Інженерія верифікації і валідації (V&V) електросамоката як 33-тя engineering axis: verification-validation meta-axis — IEEE 1012:2016 + ISO/IEC/IEEE 29119 + 12207:2017 + 15288:2015 + IEEE 730 + 1028 + V-Model + W-Model + Boehm 1979 + IV&V + ISO 26262-8 + DO-178C
У серії інженерного гайду ми описали акумуляторну батарею з BMS і thermal runaway intro, гальмівну систему, мотор і контролер, підвіску, шини, світло і видимість, раму й вилку, display + HMI, зарядний пристрій SMPS CC/CV, connector + wiring harness, IP-захист, bearingи з ISO 281 L10, стеблину і механізм складання, деку, handgrip + lever + throttle, колесо як assembly, інженерію різьбових з’єднань як joining-axis, термоменеджмент як heat-dissipation axis, EMC/EMI як interference-mitigation axis, кібербезпеку як interconnect-trust axis, NVH як acoustic-vibration-emission axis, функціональну безпеку як safety-integrity axis, інженерію життєвого циклу батареї як sustainability axis, ремонтопридатність як repairability-axis, environmental robustness як environmental-conditioning axis, privacy і захист персональних даних як privacy-preservation axis, інженерію надійності як reliability-prediction meta-axis, software & firmware engineering як SW-process axis, human factors і ергономіку як human-machine fit axis, інженерію якості виробництва як manufacturing-process axis та інженерію управління ризиками як risk-anticipation meta-axis. Ці 32 engineering-axes описали підсистеми, способи з’єднання, теплові й електромагнітні явища, безпеку, sustainability, ремонтопридатність, environmental conditioning, privacy, reliability prediction, SW-process, human-machine fit, manufacturing process та risk anticipation. Risk-management engineering (EV) описала, як ідентифікувати + аналізувати + оцінювати + обробляти + моніторити риски. Але це лишає окреме питання: як переконатись, що те, що ми реально побудували, відповідає (а) тому, як ми сказали, що його будуватимемо (verification), і (б) тому, що насправді потрібне користувачу (validation)? Жодна з 32 попередніх axes це питання прямо не закрила.
Verification & Validation (V&V) engineering — це verification-validation meta-axis усього e-самоката. Вона надає process-and-task стандарт (IEEE 1012:2016 Standard for System, Software, and Hardware Verification and Validation — V&V life cycle процеси для systems + software + hardware з minimum V&V tasks для кожного з 4 integrity levels), testing-standard family (ISO/IEC/IEEE 29119 з 5 частин: Part 1:2022 concepts/definitions; Part 2:2021 test processes; Part 3:2021 test documentation замість withdrawn IEEE 829-2008; Part 4:2021 test techniques; Part 5:2024 keyword-driven testing), life-cycle V&V scaffolding (ISO/IEC/IEEE 12207:2017 для software + 15288:2015 для system), SQA plan-and-process (IEEE 730:2014 Software Quality Assurance Plan), review/inspection methodology (IEEE 1028:2008 з 5 типами reviews + Fagan inspection IBM 1976 origin), conceptual frameworks (V-Model Forsberg-Mooz 1991 + W-Model parallel-V&V extension), семінальне дихотомічне розрізнення (Boehm 1979 paper Guidelines for Verifying and Validating Software Requirements and Design Specifications: «Are we building the product right?» = verification; «Are we building the right product?» = validation), independence framework (IV&V — Independent V&V per IEEE 1012 з 3 independencies: technical + managerial + financial), і cross-link до high-assurance domain-specific standards (ISO 26262-8:2018 clauses 9-10 automotive software verification; DO-178C software considerations in airborne systems with 5 software levels A-E і 71 objective).
Це тридцять третя engineering-axis deep-dive у серії гайду — і шоста process meta-axis (паралельна до reliability-prediction EN + SW-process EP + human-machine-fit ER + manufacturing-process ET + risk-anticipation EV, тепер verification-validation EX). Як і всі п’ять попередніх process meta-axes, V&V axis не має «залізної» реалізації — це methodology, що визначає, як systematically довести, що 32 попередніх axes виконуються на практиці. Без V&V специфікації reliability MTTF, ALARP risk tolerance, ASIL C controllability rating, GDPR Art. 35 DPIA conclusion, Cpk ≥ 1.67 manufacturing capability, ANSUR P5-P95 ergonomic fit і навіть IP67 IPX claim лишаються paper claims — не engineering evidence.
1. V&V ≠ risk management ≠ FMEA: окрема axis
Risk management (axis EV) ідентифікує + аналізує + оцінює + обробляє риски. FMEA (axis EN inductive + ET PFMEA) каталогізує failure modes + effects. HARA (axis ED), TARA (axis DZ), і DPIA (axis EL) — це domain-specific risk-assessments. V&V engineering задає окрему methodology, що відповідає на двa principal questions, які жодна з вище не закриває:
- Verification — «Have we built the product according to the specification + requirements + risk-treatments we agreed to?» Це stage-by-stage conformance check проти detailed input artifact (requirements + design + risk-treatment plan + safety case).
- Validation — «Does the built product actually satisfy real-world user need + intended use environment?» Це end-to-end fitness-for-purpose check проти real user need, з real users у real environment.
| Вимір | Reliability FMEA (EN) | Functional safety HARA (ED) | Risk management (EV) | V&V (EX) |
|---|---|---|---|---|
| Тригер | Component reliability allocation | ISO 26262 compliance | Strategic decision / project initiation | Stage gate / acceptance |
| Output | RPN / AP per component | ASIL per hazard | Risk register + treatment plan | V&V report (test results + reviews + traceability) |
| Standard | IEC 60812:2018 | ISO 26262:2018 part 3 | ISO 31000:2018 + ISO/IEC 31010:2019 | IEEE 1012:2016 + ISO/IEC/IEEE 29119 family |
| Question | What failure modes exist? | What hazards exist? | What risks exist + how to treat? | Did we build it right? Did we build the right thing? |
| Granularity | Component | Vehicle function | Enterprise + project + operational | Each life-cycle stage + each integrity level |
Boehm 1979 seminal distinction — Barry Boehm у Guidelines for Verifying and Validating Software Requirements and Design Specifications (Proceedings EURO IFIP 79, 1979) формалізував раніше неясне розрізнення: verification — це Have we transformed input A correctly into output B? (e.g., requirements → design done right? design → code done right? code → tests done right?), тоді як validation — це Is the resulting B what user actually needs? (e.g., software does what user asked, незалежно чи через requirements/design/code path). Verification — це internal-consistency перевірка; validation — це external-need перевірка.
Risk-management engineering (EV) каже, що перевіряти (приоритезує high-risk areas). V&V engineering (EX) каже, як перевіряти (specific tasks per integrity level, specific test techniques, specific documentation). FMEA дає список failure modes — V&V каже, які тести довести, що ці failure modes не активуються або mitigation працює. HARA дає ASIL — V&V каже, скільки + які саме тести треба для досягнення необхідної confidence per ASIL.
2. Boehm 1979 — verification vs validation seminal distinction
Barry Boehm, тоді у TRW (пізніше засновник USC CSE — Center for Systems and Software Engineering), у 1979 paper зафіксував раніше плутану термінологію двома лаконічними питаннями:
| Concept | Question | Focus |
|---|---|---|
| Verification | «Are we building the product right?» | Internal consistency: did we correctly transform requirement → design → code → tested binary? |
| Validation | «Are we building the right product?» | External fitness: does the product actually satisfy stakeholder need + intended use? |
Ілюстрація для e-самоката:
- Verification step — «Specification вимагає brake-disc діаметр 140 mm ± 0.5 mm. Did the built prototype’s measured brake disc fall within 140 ± 0.5 mm?» Це verification: ми перевіряємо, чи build conforms до spec.
- Validation step — «User-need дослідження показало, що 95% city-riders очікують stopping distance ≤ 4 m at 20 km/h on dry pavement. Чи реальний прототип досягає цього в реальному city-riding scenario, не лабораторному?» Це validation: ми перевіряємо, чи product задовольняє user need.
Critical implication: можна мати 100% verification + 0% validation — продукт побудований точно за specs, але specs не відображають user need (e.g., self-balancing scooter perfectly built per spec, але користувачі не довіряють електронному балансу і відмовляються купувати). Або 0% verification + accidental validation — продукт випадково подобається користувачам, але не відповідає specs, що руйнує QA + safety case (spec казала ASIL C controller, реальний ASIL B — продукт працює, але safety-case недійсний).
ISO/IEC/IEEE 24765:2017 Vocabulary (продукт спільної робочої групи SC 7 ISO/IEC JTC 1 + IEEE) формалізував обидва терміни exact-quote з Boehm. Boehm дистинкція тепер default у IEEE 1012:2016 (clause 3.1.92 verification + 3.1.93 validation), ISO/IEC/IEEE 12207:2017, ISO/IEC/IEEE 15288:2015, ISO 26262, DO-178C, ISO 14971 medical, FDA QSR 21 CFR 820.30 medical devices.
3. IEEE 1012:2016 — V&V life cycle process foundation
IEEE 1012:2016 IEEE Standard for System, Software, and Hardware Verification and Validation — опублікований August 2017 як revision IEEE 1012-2012 + Corrigendum 1; sponsored IEEE Computer Society S2ESC. Це core standard для V&V life cycle, aligned з:
- ISO/IEC/IEEE 15288:2015 Systems and software engineering — System life cycle processes — system-level processes.
- ISO/IEC/IEEE 12207:2017 Systems and software engineering — Software life cycle processes — software-level processes.
IEEE 1012:2016 описує V&V tasks для кожного process group з 15288/12207:
- Acquisition + Supply — verify contract conforms; validate supplier output.
- Concept — verify concept against need; validate need against stakeholders.
- Requirements — verify requirements consistent + complete + testable; validate requirements proti user need.
- Design — verify design conforms to requirements + design rules; validate design will satisfy intended use.
- Implementation — verify code conforms to design + coding standards; validate code’s emergent behavior.
- Integration — verify integrated components produce specified behaviors; validate integrated system satisfies user need.
- Validation — final acceptance test.
- Installation/Checkout — verify installed system behaves correctly in target environment.
- Operation + Maintenance — verify maintenance changes preserve requirements; validate ongoing fitness.
- Disposal — verify decommissioning task list executed.
Кожна V&V task має mandatory inputs, mandatory outputs, minimum tasks per integrity level (наступний розділ), optional tasks для tailoring + intensity/rigor scaling. Key V&V outputs:
- V&V Plan (VVP) — overall planning, aligned з SQA plan IEEE 730:2014.
- V&V Reports — per-stage results.
- V&V Anomaly Report — anomalies discovered; classified per IEEE 1044:2009 (severity + classification + lifecycle).
- V&V Activity Summary — gate-passage evidence для stage acceptance.
4. IEEE 1012 integrity levels 1-4 — risk-graduated V&V rigor
IEEE 1012:2016 (clause 6 + Annex C) maps consequence + likelihood до integrity level 1-4, де integrity level 1 — найнижча consequence + likelihood combination (light V&V), integrity level 4 — найвища (catastrophic + reasonable: most intensive V&V).
| Integrity level | Consequence | Likelihood | Тип системи | V&V rigor |
|---|---|---|---|---|
| 4 (high) | Catastrophic | Reasonable / probable | Safety-critical: aviation, medical implants, nuclear control | Most intensive: 100% MC/DC + formal methods + IV&V mandatory + independent analyses |
| 3 | Critical | Occasional | Mission-critical: automotive ASIL C/D, train protection systems | Intensive: structural-coverage analysis + comprehensive test + IV&V suggested |
| 2 | Marginal | Infrequent / occasional | Commerce: e-самокат BMS + brake controller + low-ASIL automotive | Moderate: unit + integration test + traceability + reviews |
| 1 (low) | Negligible | Infrequent | Non-safety/non-business-critical: e-самокат app UI + non-critical display | Light: smoke test + review |
Likelihood scale: reasonable / probable / occasional / infrequent. Consequence scale: catastrophic / critical / marginal / negligible. 4×4 matrix maps до 4 integrity levels — high-cell combinations (catastrophic + reasonable) → integrity level 4; low-cell combinations (negligible + infrequent) → integrity level 1.
Для e-самоката typical assignment:
- BMS firmware + thermal-runaway protection → IL 3 (critical consequence — fire/explosion; occasional likelihood — abuse + manufacturing defect tail).
- Brake-controller firmware (regenerative + ABS-like) → IL 3 (critical — crash risk; occasional — wet pavement + tire-grip variability).
- Display/HMI firmware → IL 2 (marginal — wrong speed display); IL 3 if display drives controllability (rider relies on speedometer для safe-speed limit обу controllers can lock-out).
- App + cloud (telemetry + remote-unlock) → IL 1 (negligible direct consequence) or IL 2 (data + privacy concern per GDPR).
- Bootloader + secure-boot crypto → IL 3 (critical — pwned controller can crash rider).
Integrity-level assignment аналогічно ASIL determination (axis ED HARA) і CAL determination (axis DZ TARA), але covers а wider scope (system + software + hardware combined) і drives V&V task set, не SI architecture. Cross-axis: HARA-determined ASIL для brake-controller → IL 3 у IEEE 1012 termini → mandatory tasks: requirements verification + design verification + 100% statement + branch coverage + integration test + system test + acceptance test + traceability + V&V independence per Annex C.
5. IV&V — Independent V&V з 3 independencies
IEEE 1012:2016 Annex G формалізує Independent V&V (IV&V) як V&V проведений організацією технічно, управлінсько, і фінансово незалежною від developer. Це критичний risk-mitigation control, бо in-house V&V має systematic confirmation bias (тестувальник той самий, хто писав код; тестувальник звітує менеджеру, чиї bonus залежить від on-time delivery; тестувальний бюджет залежить від developer cost-center).
Три independencies (вимірюються per criteria):
- Technical independence — IV&V team не використовує developer’s tools, тестових даних, simulation models; має автономно reverse-engineer + verify через independent analyses. Це знімає circular-validation risk («написав код для requirement A, написав тест для code A → тест пройшов, але requirement A неправильна»).
- Managerial independence — IV&V team звітує до stakeholder, відмінного від developer’s manager, з authority to halt developer work-stream. Це знімає resource-pressure risk («менеджер не дозволив проводити failing test, бо це коштувало б релізу»).
- Financial independence — IV&V budget — окрема line-item, не reallocateable до developer. Це знімає cost-cutting risk («бюджет IV&V вирізали для забезпечення developer scope»).
Класичні приклади IV&V:
- NASA IV&V Facility (Fairmont, West Virginia) — flagship govt IV&V program, заснована 1993 після Challenger + Hubble спірального дзеркала incidents; IV&V для Space Shuttle + ISS + Mars rovers + JWST + Artemis.
- FAA DERs (Designated Engineering Representatives) — для civil aviation per DO-178C — third-party engineering review.
- EU Notified Bodies для CE marking — third-party certification для high-risk products (Class IIa+ medical devices, MID measuring instruments).
- TÜV Rheinland / TÜV SÜD / DEKRA / UL / Intertek / SGS / Bureau Veritas — global Testing-Inspection-Certification (TIC) industry надає third-party V&V services для type approval, product certification, safety standards compliance.
Для e-самокат manufacturer: IV&V зазвичай partial (зазвичай не повна independent IV&V для consumer electronics — це financially prohibitive). Common pattern — внутрішня V&V team звітує до Quality director окремо від Engineering director (managerial independence ≥ partial); critical safety + certification testing виконано accredited 3rd-party lab per ISO/IEC 17025 (technical independence ≥ partial); third-party test costs у окремому certification cost-center (financial independence ≥ partial).
6. ISO/IEC/IEEE 12207:2017 + 15288:2015 — V&V у life cycle
ISO/IEC/IEEE 12207:2017 Systems and software engineering — Software life cycle processes + ISO/IEC/IEEE 15288:2015 Systems and software engineering — System life cycle processes — це process-reference standards, що визначають all activities of software / system life cycle, кожна з input + output + outcome + activity + task descriptions. V&V — це explicit process в обох:
- 15288:2015 clause 6.4.9 Verification process + 6.4.11 Validation process — system-level.
- 12207:2017 clause 6.4.9 Verification process + 6.4.11 Validation process — software-level.
Обидва запитують ідентичну structure:
- Purpose — provide evidence that… (verification: «system/software complies with requirements»; validation: «system/software satisfies intended use»).
- Outcomes — list of artifacts після успіху.
- Activities and tasks — список detailed activities + tasks (e.g., «prepare strategy», «identify constraints», «conduct V&V», «report results»).
12207/15288 + 1012 у co-aligned trio: 12207/15288 каже «що saved process activity», 1012 каже «як зробити це process activity з якими input/output/tasks»; IEEE 730:2014 каже «як це планувати у SQA plan». Кожна activity у 12207/15288 V&V process має corresponding task у 1012, з task description, inputs, outputs.
Для e-самоката: process flow зазвичай tailored на основі consumer electronics scale + integrity level — не повний 12207 + 15288 (це для critical-mass infrastructure). Common tailoring — Stripped-down V-Model з 4 stages (requirements → design → code → test) замість full 9-stage life cycle, з V&V у кожному gate.
7. ISO/IEC/IEEE 29119 family — five-part testing standard
ISO/IEC/IEEE 29119 — пʼятичастинний testing-standard family, joint SC 7 / IEEE Computer Society робота, що replaces withdrawn IEEE 829-2008 (Test Documentation) + IEEE 1008-1987 (Software Unit Testing) + BS 7925-1/2 / IEEE 1059-1993. Latest revisions:
| Part | Назва | Latest | Замінює |
|---|---|---|---|
| Part 1 | Concepts and definitions | 2022-01 (replaces 2013) | вокабуляр testing |
| Part 2 | Test processes | 2021-10 | test management + test design + test execution processes |
| Part 3 | Test documentation | 2021-10 | IEEE 829-2008 |
| Part 4 | Test techniques | 2021-10 | BS 7925-2 |
| Part 5 | Keyword-driven testing | 2024 (replaces 2016) | (новий area) |
Part 1:2022 Concepts and definitions — 70+ terms нормативної vocabulary (test case, test design, test item, test object, test oracle, test procedure, test script, test suite, test report, test policy, test strategy, test plan, test design specification, test approach, test environment, test condition, test data, test execution, test log, test incident, test result), aligned з ISO/IEC/IEEE 24765 vocabulary.
Part 2:2021 Test processes — three layers:
- Organizational test process — sets test policy + test strategy.
- Test management processes — test planning + test monitoring/control + test completion (per project / per release).
- Dynamic test processes — test design + test implementation + test execution + test incident reporting (per test cycle).
Part 3:2021 Test documentation — templates + content elements для 19 типів artifacts: test policy, organizational test strategy, project test plan, test strategy, test design spec, test case spec, test procedure spec, test data requirements, test environment requirements, test data readiness report, test environment readiness report, actual results, test result, test execution log, test incident report, test completion report. Replaces IEEE 829-2008.
Part 4:2021 Test techniques — catalogue:
- Specification-based (black-box): equivalence partitioning, BVA (boundary value analysis), decision table testing, cause-effect graphing, state transition testing, syntax testing, scenario testing, use-case testing, classification tree method (CTM), combinatorial / pairwise testing, random testing.
- Structure-based (white-box): statement testing, branch testing, decision testing, condition testing, MC/DC, multiple condition testing, data flow testing, path testing.
- Experience-based: error guessing, exploratory testing, checklist-based testing.
Part 5:2024 Keyword-driven testing — abstraction layer for test scripting, де test cases описані як sequences of keywords (high-level actions) replaceable across products + tools.
8. Test coverage criteria — statement / branch / MC/DC / path
Coverage criteria — formalized у Myers Art of Software Testing (1979) + ISO/IEC/IEEE 29119-4:2021 — defines completeness criteria для test suite. Hierarchy of strictness:
| Criterion | Що покривається | Example | Use |
|---|---|---|---|
| Statement | Кожен executable statement у код виконано ≥ 1 раз | if (x>0) y=1; — 1 test з x>0 покриває обидва statement | Bare minimum; не сильна |
| Branch (decision) | Кожна decision (if/while/for/case) бере both true + false outcomes | if (x>0) y=1; — потрібно 2 tests (x>0 + x≤0) | DO-178C Level B + ISO 26262 ASIL B mandatory |
| Condition | Кожна boolean condition бере both true + false | if (x>0 && y<5) — 2 tests per condition (x>0/x≤0; y<5/y≥5) | Stronger than branch |
| MC/DC | Modified Condition/Decision Coverage — each condition independently affects decision outcome | if (A && (B || C)) — потрібно 4+ tests showing each of A, B, C independently affecting outcome | DO-178C Level A mandatory; ISO 26262 ASIL D mandatory |
| Multiple condition | Each combination of conditions tested | if (A && B) — 4 tests (A∧B, A∧¬B, ¬A∧B, ¬A∧¬B) | Exhaustive but combinatorial explosion |
| Path | Each possible execution path тестується | NP-hard для loops; practical alternative — basis path testing (McCabe) | Research / very-high-assurance |
MC/DC (Modified Condition/Decision Coverage) — формалізовано John Chilenski + Steven Miller (NASA Boeing) 1994 paper Applicability of Modified Condition/Decision Coverage to Software Testing; required by DO-178C для Level A software + by ISO 26262-6:2018 Table 12 для ASIL D unit verification. Definition: each condition в decision is shown to independently affect that decision’s outcome — для умови з n conditions достатньо n+1 test cases (vs 2^n для multiple-condition coverage).
Для e-самоката BMS firmware (IL 3 / ASIL B): target branch + decision coverage 100%; для critical-path cell-voltage trip-logic — MC/DC 100% recommended. Coverage measured автоматичним instrumentation tool (gcov, LDRA Testbed, VectorCAST, Cantata, BullseyeCoverage).
9. Test techniques — equivalence partitioning + BVA + decision tables
Equivalence partitioning (EP) — domain поділено на equivalence classes, де всі члени класу очікувано викликають однакову system response. Test ≥ 1 representative per class. Зменшує # tests з infeasible to manageable.
Boundary Value Analysis (BVA) — bugs cluster біля boundaries; test on boundary + just-inside + just-outside. Семінально Myers 1979.
Decision tables — exhaustively map combinations of input conditions до output actions; useful для business-rules з multiple AND/OR conditions.
State transition testing — model system як FSM (states + transitions + guards + events + actions); test all states reachable + all transitions traversed + invalid transitions rejected. Critical для controllers/HMI/protocols.
Cause-effect graphing — formalize logical relationship causes → effects; derive decision-table coverage.
Classification tree method (CTM) — hierarchical decomposition of input domain into orthogonal classifications; combine via combinatorial techniques.
Pairwise / orthogonal-array testing — для n-factor input combinatorial explosion: test всі pairs combinations (not всі n-tuples); typically 2-7% of full Cartesian, виявляє ~70-90% defects per empirical studies (Kuhn-Wallace-Gallo 2004 NIST).
Fuzzing (random / property-based) — generate random or constrained-random inputs; useful для error-handling robustness. Property-based — define invariants, generator probes для counter-examples (QuickCheck Haskell 2000; PropEr Erlang; Hypothesis Python; PropTest Java).
Приклад для e-самокат BMS overcurrent test:
- EP: classes {I < 0 (regen), 0 ≤ I ≤ I_nominal, I_nominal < I ≤ I_limit, I > I_limit (overcurrent)}.
- BVA: тести при I = -1 A, 0 A, I_nominal − ε, I_nominal, I_nominal + ε, I_limit − ε, I_limit, I_limit + ε.
- State transition: states {normal, overcurrent_detected, overcurrent_tripped, lockout}; transitions on threshold crossings + recovery events; verify wrong-direction transitions rejected.
- Fuzzing: random current profiles 1000-runs; verify trip-time < spec (e.g., 100 ms) для each high-current excursion.
10. Reviews + inspections — IEEE 1028:2008 + Fagan
IEEE 1028:2008 IEEE Standard for Software Reviews and Audits визначає 5 типів reviews/audits, кожен з нормативною procedure + roles + outputs:
| Type | Purpose | Formality | Output |
|---|---|---|---|
| Management review | Status + risks + decisions | Low; informal | Action items |
| Technical review | Technical adequacy | Medium | Recommendation + defect list |
| Inspection (Fagan) | Defect detection | High; trained roles | Defect log + metrics |
| Walk-through | Educate + find issues | Low-medium | Notes |
| Audit | Compliance | High; formal | Audit report |
Fagan inspection — formalized Michael Fagan IBM 1976 paper Design and Code Inspections to Reduce Errors in Program Development (IBM Systems Journal Vol. 15 No. 3). Це structured peer review process з 6 stages: planning → overview → preparation → inspection meeting → rework → follow-up. Roles: moderator + reader + recorder + inspectors (3-6). Empirical data — Fagan inspection виявляє 60-90% defects (vs 30-40% для informal review + 20-50% для testing alone). Cost: 1-2 hours preparation per inspector + 2-hour meeting per 200-400 LoC.
Для e-самокат safety-critical code (BMS, brake controller, ASIL B+): Fagan inspection mandatory для critical-path code (e.g., overcurrent-trip logic; brake-pedal-input-to-motor-cutoff path). Less critical code — peer code review через Git PR з ≥ 1 approver + checklist (replacing formal Fagan з ~10× less defect-finding capacity, але lower friction).
11. V-Model + W-Model — life-cycle visualization
V-Model — формалізовано Kevin Forsberg + Harold Mooz у 1991 paper The Relationship of System Engineering to the Project Cycle (Proceedings 1st Annual Symposium of the National Council on Systems Engineering). Boehm pre-shadowed concept у Software Engineering Economics 1981. Подальша formalization — Andreas Spillner 2002 + IABG (German DoD) Bundeswehr V-Modell XT 2005.
Структура — V-shape:
Requirements ---------------> Acceptance test
\ /
Architecture ----> System test
\ /
Design ----> Integration test
\ /
Code (bottom of V)
↕
Unit test
Left side — decomposition (top-down): user need → requirements → architecture → design → code. Right side — integration + verification (bottom-up): unit test → integration test → system test → acceptance test. Each right-side stage verifies the corresponding left-side stage.
W-Model — extension by Andreas Spillner 2002, акцентує parallel V&V activities для кожного development stage rather than after-the-fact. Замість V (V&V лише після code), W має дві V’s:
- First V — development (requirements → … → code).
- Second V (W’s right peak) — V&V activities у parallel: requirements V&V (review + traceability + testability check) → design V&V (review + design analyses) → code V&V (static analysis + reviews) → testing.
W-Model captures shift-left testing — defect found at requirements stage 10-100× cheaper to fix than after release (Capers Jones data: $25 per defect at requirements review vs $16,000 at field release for typical software).
12. Traceability matrix — RTM
Requirements Traceability Matrix (RTM) — N×M matrix where rows = requirements, columns = artifacts (design elements, code modules, test cases, risk-treatments). Cell = «requirement i covered by artifact j».
Forward traceability — requirement → design → code → test. Answers: «Is this requirement implemented + tested?»
Backward traceability — test → code → design → requirement. Answers: «What requirement does this test cover?»
Bidirectional traceability — обидва. Required by IEEE 1012:2016 + ISO 26262 + DO-178C + ISO 14971 + FDA 21 CFR 820.30.
Для e-самоката BMS firmware:
| Req ID | Requirement | Design ref | Code ref | Test ref | Risk-treatment |
|---|---|---|---|---|---|
| BMS-REQ-001 | BMS shall trip output relay within 100 ms of cell voltage exceeding 4.25 V | DSGN-BMS-OV §2.3 | bms.c::check_overvoltage() | TC-BMS-001 + TC-BMS-002 (BVA) | EV-RISK-007 thermal-runaway mitigation |
| BMS-REQ-002 | BMS shall measure all 13 cell voltages at ≥ 100 Hz | DSGN-BMS-ADC §3.1 | bms.c::sample_cells() | TC-BMS-005 + TC-BMS-006 | EV-RISK-007 |
| … | … | … | … | … | … |
Cross-axis: RTM зв’язує risk-management EV (risk-treatment IDs) до V&V EX (test IDs) до functional safety ED (safety-requirement IDs) до reliability EN (FMEA-derived requirement IDs). Без RTM, claim «we implemented + tested все critical» — unverifiable. TIC labs (TÜV, DEKRA, UL) always demand RTM for safety certification.
13. Risk-based testing + mutation testing + regression
Risk-based testing (RBT) — ISO/IEC/IEEE 29119-2:2021 clause 5 — prioritize test design + execution by risk-management EV output. High-risk items → more thorough V&V (more techniques, deeper coverage criteria, IV&V); low-risk → lighter. Pragmatic, бо повне V&V кожного requirement infeasible.
RBT decision flow:
- Identify risks from risk register (EV output) — software failures, integration breaks, requirement misinterpretations.
- Score likelihood + impact.
- Allocate test effort proportional to risk score.
- Select techniques based on risk type — boundary risks → BVA; combinatorial risks → pairwise; protocol risks → state-transition.
Mutation testing — formalized Richard DeMillo + Richard Lipton + Frederick Sayward 1978 paper Hints on Test Data Selection: Help for the Practicing Programmer (IEEE Computer Vol. 11 No. 4). Idea: generate mutants (small syntactic changes до code, e.g., < → ≤); run test suite; mutation score = (killed mutants) / (total non-equivalent mutants). Low score = test suite weak (not exercising potentially bug-relevant changes).
Two underlying hypotheses:
- Competent programmer hypothesis — programmers write code that’s close to correct; small mutations approximate real bugs.
- Coupling effect hypothesis — test suite that catches small simple mutations also catches complex deeper bugs.
Tools: PIT (Java), Stryker (JS/TS/C#/Scala), mutmut (Python), Mull (C/C++). For e-самокат firmware: target mutation score ≥ 75-80% для safety-critical code.
Regression testing — after change, rerun previously-passing tests to ensure нічого не зломалось. Subcategories:
- Smoke testing — minimal test set, confirms build basically works.
- Sanity testing — narrow focus on specific change area.
- Full regression — entire test suite; expensive but comprehensive.
Test selection strategies для regression: dependency-based (only run tests touching changed code via static analysis), risk-based (recurring + high-risk areas), time-budgeted (всі critical + as much full as fits in 8 hours).
14. ISO 26262-8:2018 + DO-178C — cross-industry V&V
ISO 26262-8:2018 Road vehicles — Functional safety — Part 8: Supporting processes — automotive functional-safety V&V:
- Clause 9 Verification — verification of safety requirements + safety architecture + technical safety concept.
- Clause 10 Software verification — code review + static analysis + structural coverage + functional testing per ASIL.
- Clause 11 Confidence in the use of software tools — TCL (Tool Confidence Level) determination + tool qualification.
- Cross-link до ISO 26262-6:2018 — software unit verification (clause 9 — Table 12 coverage requirements: ASIL A statement; ASIL B branch; ASIL C MC/DC; ASIL D 100% MC/DC + control + data flow); software integration testing (clause 10 — back-to-back simulation тестування ECU + MIL/SIL/PIL/HIL).
DO-178C Software Considerations in Airborne Systems and Equipment Certification — RTCA/EUROCAE 2011 standard (replaces DO-178B 1992); FAA Advisory Circular AC 20-115C recognizes; EASA recognizes. Defines 5 software levels (DAL — Design Assurance Level) A-E, кожен з different objectives:
| DAL | Failure condition | Objectives | Software example |
|---|---|---|---|
| A | Catastrophic — loss of aircraft + lives | 71 з 71 з MC/DC | Flight control computer |
| B | Hazardous | 69 of 71; decision coverage | Autopilot |
| C | Major | 62 of 71; statement coverage | Cabin pressurization |
| D | Minor | 26 of 71 | Cabin entertainment |
| E | No safety effect | 0 з 71 | Inflight WiFi UI |
Companions: DO-254 (hardware), DO-330 (tool qualification), DO-331 (model-based development), DO-332 (object-oriented), DO-333 (formal methods).
Cross-industry transfer для e-самоката: BMS firmware controlling battery safety = automotive ASIL B → IEEE 1012 IL 3 → ISO 26262-6 Table 12 ASIL B → branch coverage 100% + decision coverage 100% + integration tests + functional tests; no MC/DC mandate unless ASIL escalates. Brake controller (regenerative) — similar.
15. 32-row cross-axis matrix — V&V relevance до 32 prior axes
V&V engineering — meta-axis above 32 попередніх. Для кожної axis існує specific V&V activity that converts spec claim into engineering evidence:
| # | Engineering axis | V&V activity / artifact |
|---|---|---|
| 1 | Battery + BMS (DC) | Cycling chamber (IEC 62660-1 / UN 38.3 T.1-T.8 tests); HVAC chamber; thermal-runaway propagation test (UL 9540A) |
| 2 | Brake system (DE) | Brake dyno (SAE J2522 fade); cold/wet stop test; ABS HiL |
| 3 | Motor + controller (DG) | Dyno (torque-speed-current MAP); HiL torque-loop verification; UNECE R85 NetPower |
| 4 | Suspension (DI) | Vehicle dyno; bump-track; payload-cycle endurance test |
| 5 | Tires (DK) | UNECE R75; wet-grip; rolling-resistance (ISO 28580); endurance |
| 6 | Lighting (DM) | Photometric goniometer; SAE J583 / ECE R3; chamber-aged degradation |
| 7 | Frame + fork (DO) | Fatigue test (EN 17128 endurance); shock-drop; FE-validation |
| 8 | Display + HMI (DQ) | Sunlight-readability lux test; touch-target ergonomic test; eye-tracking (NHTSA glance) |
| 9 | Charger SMPS (DS) | EMC chamber (CISPR 14); thermal cycling; IEC 60068-2 sample-aged |
| 10 | Connector + harness (DU) | Pull-force; cyclic mating; salt spray; voltage drop; FE-validation |
| 11 | IP захист (DW) | IPX chamber (IEC 60529); dust chamber; vibration combined |
| 12 | Bearings (DY) | L10 cycle; vibration; dimensional QA; pre-shipment burnin |
| 13 | Stem + folding (EA) | Locking-force cycle; vibration; safety-stop test |
| 14 | Deck (EC) | Flexural strength FE-validated; impact-energy test |
| 15 | Handgrip + throttle (EE) | Cycle-test (≥ 100 k operations); LED contrast (ISO 9241-303) |
| 16 | Wheel assembly (EG) | Static + dynamic load; spoke tension; runout |
| 17 | Fastener + bolted joint (DT) | Torque-tension test (axial + transverse); vibration loosening; salt-spray |
| 18 | Thermal management (DV) | Pull-down chamber; IR thermography; NTC sensor calibration |
| 19 | EMC/EMI (DX) | Anechoic chamber (CISPR 32; UNECE R10) for emissions + immunity |
| 20 | Cybersecurity (DZ) | Penetration test; fuzzing of BLE/OBD/CAN; secure-boot bypass attempts; TARA scenarios |
| 21 | NVH (EB) | Sound-power semi-anechoic chamber (ISO 3744); accelerometer-array; FFT |
| 22 | Functional safety (ED) | HARA evidence + ASIL-graduated tests; HiL; back-to-back simulation |
| 23 | Sustainability (EF) | LCA Gate-to-gate inventory; recyclability fraction; battery-passport disclosure |
| 24 | Repair (EH) | Disassembly time test; spare-parts availability; documentation completeness |
| 25 | Environmental robustness (EJ) | IEC 60068-2 combined cycle (humidity + temperature + vibration); salt-spray (ISO 9227) |
| 26 | Privacy (EL) | DPIA per GDPR Art. 35; data-flow validation; pen-test of PII exposure; cookie/consent audit |
| 27 | Reliability prediction (EN) | MTBF/MTTF empirical demonstration test; HALT; HASS; accelerated-life Arrhenius |
| 28 | SW & firmware (EP) | Unit tests + integration tests + V&V activities per IEC 61508-3 |
| 29 | Human factors (ER) | Usability test (formative + summative per ISO 9241-11); SAGAT/SAGAT-NASA-TLX measurements |
| 30 | Manufacturing quality (ET) | PPAP submission package; SPC capability demonstration Cpk ≥ 1.33; Gage R&R |
| 31 | Risk management (EV) | Risk register treatment-effectiveness V&V; LOPA IPL functional test; ALARP demonstration evidence |
| 32 | Regulatory (EU type approval, FCC, UNECE) | Type-approval test in accredited 3rd-party lab per ISO/IEC 17025 |
V&V — mеthodology, що покриває всі вище єдиним vocabulary (IEEE 1012 + ISO/IEC/IEEE 29119) + єдиним process model (V-Model/W-Model + 12207/15288 V&V processes) + єдиним documentation set (IEEE 1028 review records + 29119-3 test docs + RTM).
16. Owner-level V&V «tells» — DIY checklist
Споживач не побачить lab reports або RTM. Але є 8 проксі-tells, які корелюють з V&V depth:
- Test reports availability — manufacturer publishes test reports for type approval / FCC ID / UNECE R10 EMC. Public test report database (e.g., FCC OET для radio devices) — strong V&V signal. Vague claim «certified» без test report doc — weak.
- Certification body — TÜV Rheinland / TÜV SÜD / Intertek / UL / SGS / DEKRA / Bureau Veritas mark on product + on documentation. Tier-1 TIC partner = robust IV&V; unknown «cert ABC123» = weak.
- Independent test lab marks — UL Listed / UL Recognized / ETL / CE NoBo / FCC ID — third-party V&V evidence.
- Manufacturer field-issue track-record — recall history (NHTSA + EU RAPEX/Safety Gate + UK PSD) — manufacturer that recalls + transparently communicates field issues has effective V&V loop (їх IV&V detects + fixes; vs manufacturer with public field failures + no recall = weak V&V).
- Datasheet spec ↔ measurement traceability — if datasheet claims «brake stopping distance 4 m at 20 km/h on dry pavement», look for accompanying test report citing test method + accredited lab + sample size + measurement uncertainty. Bare claim without traceable evidence = weak V&V.
- Firmware-update CVE-disclosure record — manufacturer publicly tracking + patching CVEs (security V&V) — strong IV&V (penetration testing). No CVE record + no firmware updates = weak.
- Long-term warranty depth — manufacturer offering ≥ 2 yr warranty з explicit failure-mode coverage = confidence у reliability V&V; 6-month/1-yr only = weak V&V or weak confidence.
- Documentation completeness — full owner manual з safety warnings + maintenance procedures + spare-parts list + technical specifications + test certificate references — evidence that QA process is mature; minimal manual = weak SQA + likely weak V&V.
Yellow flags: «certified» без named certification body; «tested» без test method or lab name; spec sheet з dozens of numbers but no measurement-uncertainty disclosure; firmware-update changelog vague («bug fixes»); no recall history but no transparent issue list either.
Green flags: published test reports з accredited lab + sample size + uncertainty; recall transparency з RCA detail; warranty з documented MTTF/MTBF; CVE-disclosure timeline; certification marks UL/TÜV/etc.; comprehensive owner manual + Hazardous Substances declaration + WEEE marking + chemical-disclosure per REACH/RoHS.
17. Future axes — куди axis-серія розширюватиметься
Visible future axes:
- Production logistics + supply-chain security — ISO 28000:2022 Security and resilience — Security management systems; C-TPAT (Customs-Trade Partnership Against Terrorism); AEO (Authorized Economic Operator); UFLPA (Uyghur Forced Labor Prevention Act US 2021); EU Forced Labor Regulation 2024.
- Configuration management — ISO 10007:2017 Quality management — Guidelines for configuration management; IEEE 828:2012 software configuration management; CMII (Configuration Management II); baseline + change-control + status accounting + audit. Foundation для управління версіями HW + SW + firmware.
- Project management — ISO 21500:2021 Project, programme and portfolio management — Context and concepts; PMBOK 7th ed. 2021 (PMI); PRINCE2 6th ed. 2017 (Axelos); Agile + Scrum (PMI-DASSM, Scrum.org); IPMA-ICB4.
- Sustainability impact assessment — ISO 14040:2006 + ISO 14044:2006 LCA Life cycle assessment — Principles + framework + Requirements; ISO 14025:2006 Type III environmental declarations (EPD); ILCD International Reference Life Cycle Data System (EU JRC); product carbon footprint ISO 14067:2018; integration з EU Ecodesign for Sustainable Products Regulation (ESPR) 2024.
- Maintenance engineering — EN 13306:2017 Maintenance terminology; EN 17007:2017 Maintenance process and associated indicators; IEC 60300-3-14:2004 Maintenance and maintenance support; RCM-II (Moubray) + RCMa MIL-STD-3034 + MSG-3 aerospace.
Кожна з них додасть нову axis і потенційно нову process meta-axis за тією самою структурою (deep-dive у standard + worked e-самокат приклад + cross-axis matrix + DIY checklist).
Підсумок — V&V concept-як-pattern
Що описала ця V&V engineering axis:
- V&V — verification («Are we building the product right?» — Boehm 1979) + validation («Are we building the right product?»). Verification — internal-consistency проти spec; validation — external-fitness проти user need.
- IEEE 1012:2016 — core V&V standard: aligned з ISO/IEC/IEEE 15288:2015 (system) + 12207:2017 (software); V&V tasks per life-cycle stage; 4 integrity levels (1-4) з risk-graduated rigor; IV&V з 3 independencies.
- ISO/IEC/IEEE 29119 family — five-part testing standard: Part 1:2022 vocabulary; Part 2:2021 processes; Part 3:2021 documentation (replaces IEEE 829); Part 4:2021 techniques (specification/structure/experience-based); Part 5:2024 keyword-driven.
- Coverage criteria — statement < branch < decision < condition < MC/DC < path; MC/DC mandatory для DO-178C Level A + ISO 26262 ASIL D.
- Reviews + inspections — IEEE 1028:2008 + Fagan IBM 1976: 60-90% defects detected by formal inspection vs 20-50% by testing.
- V-Model + W-Model — life-cycle visualization: V-Model decomposition + verification mirror; W-Model parallel V&V + shift-left testing.
- RTM (traceability matrix) — requirements → design → code → tests bidirectional. Required by IEEE 1012 + ISO 26262 + DO-178C.
- Risk-based testing + mutation testing + regression — RBT prioritize V&V effort by risk-management EV output; mutation testing tests the tests; regression catches change-induced breaks.
- ISO 26262-8:2018 + DO-178C — cross-industry V&V для high-assurance: ASIL coverage requirements; DO-178C 5 DALs A-E з graded objectives.
- Cross-axis matrix — V&V activity для кожної з 32 попередніх axes; converts spec claim into engineering evidence.
V&V engineering — це verification-validation meta-axis усього e-самоката. Без неї specs reliability MTTF, ALARP risk-tolerance, ASIL C controllability, GDPR DPIA conclusion, Cpk ≥ 1.67 manufacturing capability, ANSUR P5-P95 ergonomic fit, IP67 і всі інші 32-axis claims лишаються paper claims — не engineering evidence. V&V — це methodology, що converts each попередню axis з claim into evidence. Owner-як-buyer не побачить lab reports + RTM, але 8 проксі-tells (test report availability, certification body, independent lab marks, recall transparency, datasheet ↔ measurement traceability, CVE record, warranty depth, documentation completeness) корелюють з V&V depth і дозволяють differentiate manufacturer з robust V&V process від such з paper claims.