E-scooter Verification & Validation (V&V) engineering as the 33rd engineering axis: verification-validation meta-axis — IEEE 1012:2016 + ISO/IEC/IEEE 29119 + 12207:2017 + 15288:2015 + IEEE 730 + 1028 + V-Model + W-Model + Boehm 1979 + IV&V + ISO 26262-8 + DO-178C
In the engineering-guide series we covered the battery with BMS and thermal-runaway intro, the brake system, motor and controller, suspension, tires, lighting and visibility, frame and fork, display + HMI, SMPS CC/CV charger, connector + wiring harness, IP protection, bearings with ISO 281 L10, stem and folding mechanism, the deck, handgrip + lever + throttle, the wheel as an assembly, fastener engineering as the joining-axis, thermal management as the heat-dissipation axis, EMC/EMI as the interference-mitigation axis, cybersecurity as the interconnect-trust axis, NVH as the acoustic-vibration-emission axis, functional safety as the safety-integrity axis, battery-lifecycle engineering as the sustainability axis, reparability as the repairability axis, environmental robustness as the environmental-conditioning axis, privacy and personal-data protection as the privacy-preservation axis, reliability engineering as the reliability-prediction meta-axis, software & firmware engineering as the SW-process axis, human factors and ergonomics as the human-machine fit axis, manufacturing-quality engineering as the manufacturing-process axis, and risk-management engineering as the risk-anticipation meta-axis. These 32 engineering axes covered subsystems, joining methods, thermal and electromagnetic phenomena, safety, sustainability, repairability, environmental conditioning, privacy, reliability prediction, SW-process, human-machine fit, manufacturing process and risk anticipation. Risk-management engineering (EV) described how to identify + analyse + evaluate + treat + monitor risks. But that leaves a separate question: how do we make sure that what we actually built corresponds (a) to how we said we’d build it (verification), and (b) to what the user actually needs (validation)? None of the 32 prior axes addresses this question directly.
Verification & Validation (V&V) engineering is the verification-validation meta-axis of the entire e-scooter. It supplies a process-and-task standard (IEEE 1012:2016 Standard for System, Software, and Hardware Verification and Validation — V&V life-cycle processes for systems + software + hardware with minimum V&V tasks for each of 4 integrity levels), a testing-standard family (ISO/IEC/IEEE 29119, in five parts: Part 1:2022 concepts/definitions; Part 2:2021 test processes; Part 3:2021 test documentation, replacing the withdrawn IEEE 829-2008; Part 4:2021 test techniques; Part 5:2024 keyword-driven testing), life-cycle V&V scaffolding (ISO/IEC/IEEE 12207:2017 for software + 15288:2015 for systems), SQA plan-and-process (IEEE 730:2014 Software Quality Assurance Plan), a review/inspection methodology (IEEE 1028:2008 with 5 review types + Fagan inspection IBM 1976 origin), conceptual frameworks (V-Model Forsberg-Mooz 1991 + W-Model parallel-V&V extension), the seminal dichotomous distinction (Boehm 1979 paper Guidelines for Verifying and Validating Software Requirements and Design Specifications: “Are we building the product right?” = verification; “Are we building the right product?” = validation), an independence framework (IV&V — Independent V&V per IEEE 1012 with 3 independencies: technical + managerial + financial), and cross-links to high-assurance domain-specific standards (ISO 26262-8:2018 clauses 9-10 automotive software verification; DO-178C software considerations in airborne systems with 5 software levels A-E and 71 objectives).
This is the thirty-third engineering-axis deep-dive in the guide series — and the sixth process meta-axis (parallel to reliability-prediction EN + SW-process EP + human-machine-fit ER + manufacturing-process ET + risk-anticipation EV, now verification-validation EX). Like all five prior process meta-axes, the V&V axis has no “hardware” implementation — it is a methodology that defines how to systematically demonstrate that the 32 prior axes hold up in practice. Without V&V, the reliability MTTF spec, the ALARP risk tolerance, the ASIL C controllability rating, the GDPR Art. 35 DPIA conclusion, the Cpk ≥ 1.67 manufacturing capability, the ANSUR P5-P95 ergonomic fit and even an IP67 IPX claim remain paper claims — not engineering evidence.
1. V&V ≠ risk management ≠ FMEA: a separate axis
Risk management (axis EV) identifies + analyses + evaluates + treats risks. FMEA (axis EN inductive + ET PFMEA) catalogues failure modes + effects. HARA (axis ED), TARA (axis DZ), and DPIA (axis EL) are domain-specific risk assessments. V&V engineering supplies a separate methodology that answers two principal questions none of the above closes:
- Verification — “Have we built the product according to the specification + requirements + risk-treatments we agreed to?” This is a stage-by-stage conformance check against detailed input artifacts (requirements + design + risk-treatment plan + safety case).
- Validation — “Does the built product actually satisfy real-world user need + intended use environment?” This is an end-to-end fitness-for-purpose check against real user need, with real users in the real environment.
| Dimension | Reliability FMEA (EN) | Functional safety HARA (ED) | Risk management (EV) | V&V (EX) |
|---|---|---|---|---|
| Trigger | Component reliability allocation | ISO 26262 compliance | Strategic decision / project initiation | Stage gate / acceptance |
| Output | RPN / AP per component | ASIL per hazard | Risk register + treatment plan | V&V report (test results + reviews + traceability) |
| Standard | IEC 60812:2018 | ISO 26262:2018 part 3 | ISO 31000:2018 + ISO/IEC 31010:2019 | IEEE 1012:2016 + ISO/IEC/IEEE 29119 family |
| Question | What failure modes exist? | What hazards exist? | What risks exist + how to treat? | Did we build it right? Did we build the right thing? |
| Granularity | Component | Vehicle function | Enterprise + project + operational | Each life-cycle stage + each integrity level |
Boehm 1979 seminal distinction — Barry Boehm, in Guidelines for Verifying and Validating Software Requirements and Design Specifications (Proceedings EURO IFIP 79, 1979), formalised the earlier-fuzzy distinction: verification = Have we transformed input A correctly into output B? (e.g., requirements → design done right? design → code done right? code → tests done right?), while validation = Is the resulting B what the user actually needs? (e.g., does the software do what the user asked, independent of the requirements/design/code path). Verification is an internal-consistency check; validation is an external-need check.
Risk-management engineering (EV) says what to verify (it prioritises high-risk areas). V&V engineering (EX) says how to verify (specific tasks per integrity level, specific test techniques, specific documentation). FMEA gives a list of failure modes — V&V says which tests demonstrate that those failure modes don’t activate or that the mitigation works. HARA gives an ASIL — V&V says how many + which tests are needed to reach the confidence required per ASIL.
2. Boehm 1979 — the verification-vs-validation seminal distinction
Barry Boehm, then at TRW (later founder of USC CSE — Center for Systems and Software Engineering), in his 1979 paper nailed down the previously-confused terminology with two concise questions:
| Concept | Question | Focus |
|---|---|---|
| Verification | “Are we building the product right?” | Internal consistency: did we correctly transform requirement → design → code → tested binary? |
| Validation | “Are we building the right product?” | External fitness: does the product actually satisfy stakeholder need + intended use? |
Illustration for an e-scooter:
- Verification step — “The spec requires brake-disc diameter 140 mm ± 0.5 mm. Did the built prototype’s measured brake disc fall within 140 ± 0.5 mm?” This is verification: we check whether the build conforms to the spec.
- Validation step — “User-need research showed that 95% of city riders expect a stopping distance ≤ 4 m at 20 km/h on dry pavement. Does the real prototype achieve that in a real city-riding scenario, not just a lab scenario?” This is validation: we check whether the product satisfies the user need.
Critical implication: it is possible to have 100% verification + 0% validation — the product is built precisely to spec, but the spec doesn’t reflect user need (e.g., a self-balancing scooter perfectly built per spec, but users don’t trust the electronic balance and refuse to buy). Or 0% verification + accidental validation — the product accidentally pleases users but doesn’t conform to spec, which wrecks the QA + safety case (the spec called for an ASIL C controller, the real one is ASIL B — the product works, but the safety case is invalid).
ISO/IEC/IEEE 24765:2017 Vocabulary (joint work of ISO/IEC JTC 1 SC 7 + IEEE) formalised both terms with an exact quote from Boehm. The Boehm distinction is now the default in IEEE 1012:2016 (clause 3.1.92 verification + 3.1.93 validation), ISO/IEC/IEEE 12207:2017, ISO/IEC/IEEE 15288:2015, ISO 26262, DO-178C, ISO 14971 medical, FDA QSR 21 CFR 820.30 medical devices.
3. IEEE 1012:2016 — V&V life-cycle process foundation
IEEE 1012:2016 IEEE Standard for System, Software, and Hardware Verification and Validation was published in August 2017 as a revision of IEEE 1012-2012 + Corrigendum 1; sponsored by the IEEE Computer Society S2ESC. It is the core standard for the V&V life cycle, aligned with:
- ISO/IEC/IEEE 15288:2015 Systems and software engineering — System life cycle processes — system-level processes.
- ISO/IEC/IEEE 12207:2017 Systems and software engineering — Software life cycle processes — software-level processes.
IEEE 1012:2016 defines V&V tasks for each process group in 15288/12207:
- Acquisition + Supply — verify the contract conforms; validate supplier output.
- Concept — verify the concept against the need; validate the need against stakeholders.
- Requirements — verify requirements are consistent + complete + testable; validate requirements against user need.
- Design — verify the design conforms to requirements + design rules; validate the design will satisfy intended use.
- Implementation — verify code conforms to design + coding standards; validate code’s emergent behaviour.
- Integration — verify integrated components produce specified behaviours; validate the integrated system satisfies user need.
- Validation — final acceptance test.
- Installation/Checkout — verify the installed system behaves correctly in the target environment.
- Operation + Maintenance — verify maintenance changes preserve requirements; validate ongoing fitness.
- Disposal — verify the decommissioning task list was executed.
Each V&V task has mandatory inputs, mandatory outputs, minimum tasks per integrity level (next section), and optional tasks for tailoring + intensity/rigor scaling. Key V&V outputs:
- V&V Plan (VVP) — overall planning, aligned with the SQA plan IEEE 730:2014.
- V&V Reports — per-stage results.
- V&V Anomaly Report — anomalies discovered; classified per IEEE 1044:2009 (severity + classification + lifecycle).
- V&V Activity Summary — gate-passage evidence for stage acceptance.
4. IEEE 1012 integrity levels 1-4 — risk-graduated V&V rigor
IEEE 1012:2016 (clause 6 + Annex C) maps consequence + likelihood to integrity levels 1-4, where integrity level 1 has the lowest consequence + likelihood combination (light V&V), and integrity level 4 has the highest (catastrophic + reasonable: the most intensive V&V).
| Integrity level | Consequence | Likelihood | System type | V&V rigor |
|---|---|---|---|---|
| 4 (high) | Catastrophic | Reasonable / probable | Safety-critical: aviation, medical implants, nuclear control | Most intensive: 100% MC/DC + formal methods + IV&V mandatory + independent analyses |
| 3 | Critical | Occasional | Mission-critical: automotive ASIL C/D, train protection systems | Intensive: structural-coverage analysis + comprehensive test + IV&V suggested |
| 2 | Marginal | Infrequent / occasional | Commerce: e-scooter BMS + brake controller + low-ASIL automotive | Moderate: unit + integration test + traceability + reviews |
| 1 (low) | Negligible | Infrequent | Non-safety / non-business-critical: e-scooter app UI + non-critical display | Light: smoke test + review |
Likelihood scale: reasonable / probable / occasional / infrequent. Consequence scale: catastrophic / critical / marginal / negligible. The 4×4 matrix maps to the 4 integrity levels — high-cell combinations (catastrophic + reasonable) → integrity level 4; low-cell combinations (negligible + infrequent) → integrity level 1.
For an e-scooter, typical assignment:
- BMS firmware + thermal-runaway protection → IL 3 (critical consequence — fire/explosion; occasional likelihood — abuse + manufacturing-defect tail).
- Brake controller firmware (regenerative + ABS-like) → IL 3 (critical — crash risk; occasional — wet pavement + tire-grip variability).
- Display/HMI firmware → IL 2 (marginal — wrong speed display); IL 3 if the display drives controllability (the rider relies on the speedometer for safe-speed limit and the controller can lock out).
- App + cloud (telemetry + remote-unlock) → IL 1 (negligible direct consequence) or IL 2 (data + privacy concern per GDPR).
- Bootloader + secure-boot crypto → IL 3 (critical — a pwned controller can crash the rider).
Integrity-level assignment is analogous to ASIL determination (axis ED HARA) and CAL determination (axis DZ TARA), but covers a wider scope (system + software + hardware combined) and drives the V&V task set, not the SI architecture. Cross-axis: HARA-determined ASIL for the brake controller → IL 3 in IEEE 1012 terms → mandatory tasks: requirements verification + design verification + 100% statement + branch coverage + integration test + system test + acceptance test + traceability + V&V independence per Annex C.
5. IV&V — Independent V&V with 3 independencies
IEEE 1012:2016 Annex G formalises Independent V&V (IV&V) as V&V conducted by an organisation technically, managerially, and financially independent of the developer. This is a critical risk-mitigation control, because in-house V&V has systematic confirmation bias (the tester is the same person who wrote the code; the tester reports to a manager whose bonus depends on on-time delivery; the test budget depends on the developer cost-centre).
Three independencies (each measurable against criteria):
- Technical independence — the IV&V team does not use the developer’s tools, test data, or simulation models; they must autonomously reverse-engineer + verify via independent analyses. This removes the circular-validation risk (“wrote code for requirement A, wrote test for code A → test passes, but requirement A was wrong”).
- Managerial independence — the IV&V team reports to a stakeholder distinct from the developer’s manager, with authority to halt the developer work-stream. This removes the resource-pressure risk (“the manager didn’t let me run a failing test because it would have cost the release”).
- Financial independence — the IV&V budget is a separate line-item, not reallocateable to the developer. This removes the cost-cutting risk (“we cut the IV&V budget to fund the developer scope”).
Classic examples of IV&V:
- NASA IV&V Facility (Fairmont, West Virginia) — the flagship US-government IV&V program, founded in 1993 after the Challenger + Hubble spiral-mirror incidents; IV&V for the Space Shuttle + ISS + Mars rovers + JWST + Artemis.
- FAA DERs (Designated Engineering Representatives) — for civil aviation per DO-178C — third-party engineering review.
- EU Notified Bodies for CE marking — third-party certification for high-risk products (Class IIa+ medical devices, MID measuring instruments).
- TÜV Rheinland / TÜV SÜD / DEKRA / UL / Intertek / SGS / Bureau Veritas — the global Testing-Inspection-Certification (TIC) industry provides third-party V&V services for type approval, product certification, safety-standards compliance.
For an e-scooter manufacturer, IV&V is usually partial (full independent IV&V is rarely affordable for consumer electronics). The common pattern: an in-house V&V team reports to the Quality director separately from the Engineering director (managerial independence ≥ partial); critical safety + certification testing is performed in an accredited 3rd-party lab per ISO/IEC 17025 (technical independence ≥ partial); third-party test costs sit in a separate certification cost-centre (financial independence ≥ partial).
6. ISO/IEC/IEEE 12207:2017 + 15288:2015 — V&V in the life cycle
ISO/IEC/IEEE 12207:2017 Systems and software engineering — Software life cycle processes and ISO/IEC/IEEE 15288:2015 Systems and software engineering — System life cycle processes are process-reference standards that define all activities in the software / system life cycle, each with input + output + outcome + activity + task descriptions. V&V is an explicit process in both:
- 15288:2015 clause 6.4.9 Verification process + 6.4.11 Validation process — system-level.
- 12207:2017 clause 6.4.9 Verification process + 6.4.11 Validation process — software-level.
Both demand the same structure:
- Purpose — provide evidence that… (verification: “system/software complies with requirements”; validation: “system/software satisfies intended use”).
- Outcomes — list of artifacts produced on success.
- Activities and tasks — list of detailed activities + tasks (e.g., “prepare strategy”, “identify constraints”, “conduct V&V”, “report results”).
12207/15288 + 1012 form a co-aligned trio: 12207/15288 says “what the process activity is”, 1012 says “how to do that process activity, with which inputs/outputs/tasks”, and IEEE 730:2014 says “how to plan it in the SQA plan”. Each activity in the 12207/15288 V&V process has a corresponding task in 1012 with task description, inputs, outputs.
For an e-scooter: the process flow is usually tailored according to consumer-electronics scale + integrity level — it is not a full 12207 + 15288 (that’s for critical-mass infrastructure). The common tailoring is a stripped-down V-Model with 4 stages (requirements → design → code → test) instead of a full 9-stage life cycle, with V&V at each gate.
7. ISO/IEC/IEEE 29119 family — five-part testing standard
ISO/IEC/IEEE 29119 is a five-part testing-standard family, a joint SC 7 / IEEE Computer Society effort that replaces the withdrawn IEEE 829-2008 (Test Documentation) + IEEE 1008-1987 (Software Unit Testing) + BS 7925-1/2 / IEEE 1059-1993. Latest revisions:
| Part | Title | Latest | Replaces |
|---|---|---|---|
| Part 1 | Concepts and definitions | 2022-01 (replaces 2013) | testing vocabulary |
| Part 2 | Test processes | 2021-10 | test-management + test-design + test-execution processes |
| Part 3 | Test documentation | 2021-10 | IEEE 829-2008 |
| Part 4 | Test techniques | 2021-10 | BS 7925-2 |
| Part 5 | Keyword-driven testing | 2024 (replaces 2016) | (new area) |
Part 1:2022 Concepts and definitions — 70+ normative-vocabulary terms (test case, test design, test item, test object, test oracle, test procedure, test script, test suite, test report, test policy, test strategy, test plan, test design specification, test approach, test environment, test condition, test data, test execution, test log, test incident, test result), aligned with ISO/IEC/IEEE 24765 vocabulary.
Part 2:2021 Test processes — three layers:
- Organizational test process — sets test policy + test strategy.
- Test management processes — test planning + test monitoring/control + test completion (per project / per release).
- Dynamic test processes — test design + test implementation + test execution + test incident reporting (per test cycle).
Part 3:2021 Test documentation — templates + content elements for 19 artifact types: test policy, organizational test strategy, project test plan, test strategy, test design spec, test case spec, test procedure spec, test data requirements, test environment requirements, test data readiness report, test environment readiness report, actual results, test result, test execution log, test incident report, test completion report. Replaces IEEE 829-2008.
Part 4:2021 Test techniques — catalogue:
- Specification-based (black-box): equivalence partitioning, BVA (boundary value analysis), decision table testing, cause-effect graphing, state transition testing, syntax testing, scenario testing, use-case testing, classification tree method (CTM), combinatorial / pairwise testing, random testing.
- Structure-based (white-box): statement testing, branch testing, decision testing, condition testing, MC/DC, multiple condition testing, data flow testing, path testing.
- Experience-based: error guessing, exploratory testing, checklist-based testing.
Part 5:2024 Keyword-driven testing — an abstraction layer for test scripting, where test cases are described as sequences of keywords (high-level actions) replaceable across products + tools.
8. Test coverage criteria — statement / branch / MC/DC / path
Coverage criteria — formalised in Myers’ Art of Software Testing (1979) + ISO/IEC/IEEE 29119-4:2021 — define completeness criteria for a test suite. Hierarchy of strictness:
| Criterion | What is covered | Example | Use |
|---|---|---|---|
| Statement | Each executable statement in the code is executed ≥ 1 time | if (x>0) y=1; — 1 test with x>0 covers both statements | Bare minimum; not strong |
| Branch (decision) | Each decision (if/while/for/case) takes both true + false outcomes | if (x>0) y=1; — 2 tests needed (x>0 + x≤0) | DO-178C Level B + ISO 26262 ASIL B mandatory |
| Condition | Each boolean condition takes both true + false | if (x>0 && y<5) — 2 tests per condition (x>0/x≤0; y<5/y≥5) | Stronger than branch |
| MC/DC | Modified Condition/Decision Coverage — each condition independently affects the decision outcome | if (A && (B || C)) — 4+ tests needed showing each of A, B, C independently affecting the outcome | DO-178C Level A mandatory; ISO 26262 ASIL D mandatory |
| Multiple condition | Each combination of conditions is tested | if (A && B) — 4 tests (A∧B, A∧¬B, ¬A∧B, ¬A∧¬B) | Exhaustive but combinatorial explosion |
| Path | Each possible execution path is tested | NP-hard for loops; practical alternative — basis path testing (McCabe) | Research / very-high-assurance |
MC/DC (Modified Condition/Decision Coverage) — formalised by John Chilenski + Steven Miller (NASA Boeing) in their 1994 paper Applicability of Modified Condition/Decision Coverage to Software Testing; required by DO-178C for Level A software + by ISO 26262-6:2018 Table 12 for ASIL D unit verification. Definition: each condition in a decision is shown to independently affect that decision’s outcome — for a condition with n sub-conditions, n+1 test cases suffice (vs 2^n for multiple-condition coverage).
For e-scooter BMS firmware (IL 3 / ASIL B): target 100% branch + decision coverage; for the critical-path cell-voltage trip-logic — 100% MC/DC recommended. Coverage is measured by an automated instrumentation tool (gcov, LDRA Testbed, VectorCAST, Cantata, BullseyeCoverage).
9. Test techniques — equivalence partitioning + BVA + decision tables
Equivalence partitioning (EP) — the domain is split into equivalence classes, where all members of a class are expected to trigger the same system response. Test ≥ 1 representative per class. Reduces # tests from infeasible to manageable.
Boundary Value Analysis (BVA) — bugs cluster near boundaries; test on boundary + just-inside + just-outside. Seminally Myers 1979.
Decision tables — exhaustively map combinations of input conditions to output actions; useful for business-rules with multiple AND/OR conditions.
State transition testing — model the system as a FSM (states + transitions + guards + events + actions); test that all states are reachable + all transitions are traversed + invalid transitions are rejected. Critical for controllers/HMI/protocols.
Cause-effect graphing — formalise the logical relationship causes → effects; derive decision-table coverage.
Classification tree method (CTM) — hierarchical decomposition of the input domain into orthogonal classifications; combine via combinatorial techniques.
Pairwise / orthogonal-array testing — for an n-factor combinatorial explosion: test all pairs of combinations (not all n-tuples); typically 2-7% of the full Cartesian product, detecting ~70-90% of defects per empirical studies (Kuhn-Wallace-Gallo 2004 NIST).
Fuzzing (random / property-based) — generate random or constrained-random inputs; useful for error-handling robustness. Property-based — define invariants, generator probes for counter-examples (QuickCheck Haskell 2000; PropEr Erlang; Hypothesis Python; PropTest Java).
Example for e-scooter BMS overcurrent test:
- EP: classes {I < 0 (regen), 0 ≤ I ≤ I_nominal, I_nominal < I ≤ I_limit, I > I_limit (overcurrent)}.
- BVA: tests at I = -1 A, 0 A, I_nominal − ε, I_nominal, I_nominal + ε, I_limit − ε, I_limit, I_limit + ε.
- State transition: states {normal, overcurrent_detected, overcurrent_tripped, lockout}; transitions on threshold crossings + recovery events; verify wrong-direction transitions are rejected.
- Fuzzing: random current profiles for 1000 runs; verify trip-time < spec (e.g., 100 ms) for each high-current excursion.
10. Reviews + inspections — IEEE 1028:2008 + Fagan
IEEE 1028:2008 IEEE Standard for Software Reviews and Audits defines 5 types of reviews/audits, each with a normative procedure + roles + outputs:
| Type | Purpose | Formality | Output |
|---|---|---|---|
| Management review | Status + risks + decisions | Low; informal | Action items |
| Technical review | Technical adequacy | Medium | Recommendation + defect list |
| Inspection (Fagan) | Defect detection | High; trained roles | Defect log + metrics |
| Walk-through | Educate + find issues | Low-medium | Notes |
| Audit | Compliance | High; formal | Audit report |
Fagan inspection — formalised by Michael Fagan at IBM in 1976, in the paper Design and Code Inspections to Reduce Errors in Program Development (IBM Systems Journal Vol. 15 No. 3). It is a structured peer-review process with 6 stages: planning → overview → preparation → inspection meeting → rework → follow-up. Roles: moderator + reader + recorder + inspectors (3-6). Empirical data — Fagan inspection detects 60-90% of defects (vs 30-40% for informal review + 20-50% for testing alone). Cost: 1-2 hours preparation per inspector + a 2-hour meeting per 200-400 LoC.
For e-scooter safety-critical code (BMS, brake controller, ASIL B+): Fagan inspection is mandatory for critical-path code (e.g., overcurrent-trip logic; brake-pedal-input-to-motor-cutoff path). Less critical code — peer code review via Git PR with ≥ 1 approver + checklist (replaces formal Fagan at ~10× less defect-finding capacity but lower friction).
11. V-Model + W-Model — life-cycle visualisation
V-Model — formalised by Kevin Forsberg + Harold Mooz in the 1991 paper The Relationship of System Engineering to the Project Cycle (Proceedings 1st Annual Symposium of the National Council on Systems Engineering). Boehm pre-shadowed the concept in Software Engineering Economics 1981. Subsequent formalisation — Andreas Spillner 2002 + IABG (German DoD) Bundeswehr V-Modell XT 2005.
Structure — V-shape:
Requirements ---------------> Acceptance test
\ /
Architecture ----> System test
\ /
Design ----> Integration test
\ /
Code (bottom of V)
↕
Unit test
Left side — decomposition (top-down): user need → requirements → architecture → design → code. Right side — integration + verification (bottom-up): unit test → integration test → system test → acceptance test. Each right-side stage verifies the corresponding left-side stage.
W-Model — an extension by Andreas Spillner 2002, emphasising parallel V&V activities for each development stage rather than after the fact. Instead of a V (V&V only after code), W has two V’s:
- First V — development (requirements → … → code).
- Second V (W’s right peak) — V&V activities in parallel: requirements V&V (review + traceability + testability check) → design V&V (review + design analyses) → code V&V (static analysis + reviews) → testing.
The W-Model captures shift-left testing — a defect found at the requirements stage is 10-100× cheaper to fix than after release (Capers Jones data: $25 per defect at requirements review vs $16,000 at field release for typical software).
12. Traceability matrix — RTM
Requirements Traceability Matrix (RTM) — an N×M matrix where rows = requirements, columns = artifacts (design elements, code modules, test cases, risk-treatments). A cell = “requirement i is covered by artifact j”.
Forward traceability — requirement → design → code → test. Answers: “Is this requirement implemented + tested?”
Backward traceability — test → code → design → requirement. Answers: “What requirement does this test cover?”
Bidirectional traceability — both. Required by IEEE 1012:2016 + ISO 26262 + DO-178C + ISO 14971 + FDA 21 CFR 820.30.
For e-scooter BMS firmware:
| Req ID | Requirement | Design ref | Code ref | Test ref | Risk-treatment |
|---|---|---|---|---|---|
| BMS-REQ-001 | BMS shall trip output relay within 100 ms of cell voltage exceeding 4.25 V | DSGN-BMS-OV §2.3 | bms.c::check_overvoltage() | TC-BMS-001 + TC-BMS-002 (BVA) | EV-RISK-007 thermal-runaway mitigation |
| BMS-REQ-002 | BMS shall measure all 13 cell voltages at ≥ 100 Hz | DSGN-BMS-ADC §3.1 | bms.c::sample_cells() | TC-BMS-005 + TC-BMS-006 | EV-RISK-007 |
| … | … | … | … | … | … |
Cross-axis: the RTM links risk-management EV (risk-treatment IDs) to V&V EX (test IDs) to functional safety ED (safety-requirement IDs) to reliability EN (FMEA-derived requirement IDs). Without an RTM, the claim “we implemented + tested everything critical” is unverifiable. TIC labs (TÜV, DEKRA, UL) always demand an RTM for safety certification.
13. Risk-based testing + mutation testing + regression
Risk-based testing (RBT) — ISO/IEC/IEEE 29119-2:2021 clause 5 — prioritises test design + execution by risk-management EV output. High-risk items get more thorough V&V (more techniques, deeper coverage criteria, IV&V); low-risk get lighter. Pragmatic, because full V&V of every requirement is infeasible.
RBT decision flow:
- Identify risks from the risk register (EV output) — software failures, integration breaks, requirement misinterpretations.
- Score likelihood + impact.
- Allocate test effort proportional to risk score.
- Select techniques based on risk type — boundary risks → BVA; combinatorial risks → pairwise; protocol risks → state-transition.
Mutation testing — formalised by Richard DeMillo + Richard Lipton + Frederick Sayward in the 1978 paper Hints on Test Data Selection: Help for the Practicing Programmer (IEEE Computer Vol. 11 No. 4). Idea: generate mutants (small syntactic changes to the code, e.g., < → ≤); run the test suite; mutation score = (killed mutants) / (total non-equivalent mutants). A low score = the test suite is weak (it doesn’t exercise potentially bug-relevant changes).
Two underlying hypotheses:
- Competent programmer hypothesis — programmers write code that’s close to correct; small mutations approximate real bugs.
- Coupling effect hypothesis — a test suite that catches small simple mutations also catches complex deeper bugs.
Tools: PIT (Java), Stryker (JS/TS/C#/Scala), mutmut (Python), Mull (C/C++). For e-scooter firmware: target mutation score ≥ 75-80% for safety-critical code.
Regression testing — after a change, rerun previously-passing tests to ensure nothing broke. Subcategories:
- Smoke testing — minimal test set, confirms the build basically works.
- Sanity testing — narrow focus on the specific change area.
- Full regression — the entire test suite; expensive but comprehensive.
Test-selection strategies for regression: dependency-based (only run tests touching changed code via static analysis), risk-based (recurring + high-risk areas), time-budgeted (all critical + as much full as fits in 8 hours).
14. ISO 26262-8:2018 + DO-178C — cross-industry V&V
ISO 26262-8:2018 Road vehicles — Functional safety — Part 8: Supporting processes — automotive functional-safety V&V:
- Clause 9 Verification — verification of safety requirements + safety architecture + technical safety concept.
- Clause 10 Software verification — code review + static analysis + structural coverage + functional testing per ASIL.
- Clause 11 Confidence in the use of software tools — TCL (Tool Confidence Level) determination + tool qualification.
- Cross-link to ISO 26262-6:2018 — software unit verification (clause 9 — Table 12 coverage requirements: ASIL A statement; ASIL B branch; ASIL C MC/DC; ASIL D 100% MC/DC + control + data flow); software integration testing (clause 10 — back-to-back simulation testing ECU + MIL/SIL/PIL/HIL).
DO-178C Software Considerations in Airborne Systems and Equipment Certification — RTCA/EUROCAE 2011 standard (replaces DO-178B 1992); the FAA’s Advisory Circular AC 20-115C recognises it; EASA recognises it. Defines 5 software levels (DAL — Design Assurance Level) A-E, each with different objectives:
| DAL | Failure condition | Objectives | Software example |
|---|---|---|---|
| A | Catastrophic — loss of aircraft + lives | 71 of 71 with MC/DC | Flight control computer |
| B | Hazardous | 69 of 71; decision coverage | Autopilot |
| C | Major | 62 of 71; statement coverage | Cabin pressurisation |
| D | Minor | 26 of 71 | Cabin entertainment |
| E | No safety effect | 0 of 71 | In-flight Wi-Fi UI |
Companions: DO-254 (hardware), DO-330 (tool qualification), DO-331 (model-based development), DO-332 (object-oriented), DO-333 (formal methods).
Cross-industry transfer for an e-scooter: BMS firmware controlling battery safety = automotive ASIL B → IEEE 1012 IL 3 → ISO 26262-6 Table 12 ASIL B → 100% branch coverage + 100% decision coverage + integration tests + functional tests; no MC/DC mandate unless ASIL escalates. The brake controller (regenerative) — similar.
15. 32-row cross-axis matrix — V&V relevance to 32 prior axes
V&V engineering is a meta-axis above the 32 prior. For each axis there is a specific V&V activity that converts a spec claim into engineering evidence:
| # | Engineering axis | V&V activity / artifact |
|---|---|---|
| 1 | Battery + BMS (DC) | Cycling chamber (IEC 62660-1 / UN 38.3 T.1-T.8 tests); HVAC chamber; thermal-runaway propagation test (UL 9540A) |
| 2 | Brake system (DE) | Brake dyno (SAE J2522 fade); cold/wet stop test; ABS HiL |
| 3 | Motor + controller (DG) | Dyno (torque-speed-current MAP); HiL torque-loop verification; UNECE R85 NetPower |
| 4 | Suspension (DI) | Vehicle dyno; bump-track; payload-cycle endurance test |
| 5 | Tires (DK) | UNECE R75; wet-grip; rolling-resistance (ISO 28580); endurance |
| 6 | Lighting (DM) | Photometric goniometer; SAE J583 / ECE R3; chamber-aged degradation |
| 7 | Frame + fork (DO) | Fatigue test (EN 17128 endurance); shock-drop; FE-validation |
| 8 | Display + HMI (DQ) | Sunlight-readability lux test; touch-target ergonomic test; eye-tracking (NHTSA glance) |
| 9 | Charger SMPS (DS) | EMC chamber (CISPR 14); thermal cycling; IEC 60068-2 sample-aged |
| 10 | Connector + harness (DU) | Pull-force; cyclic mating; salt spray; voltage drop; FE-validation |
| 11 | IP protection (DW) | IPX chamber (IEC 60529); dust chamber; vibration combined |
| 12 | Bearings (DY) | L10 cycle; vibration; dimensional QA; pre-shipment burn-in |
| 13 | Stem + folding (EA) | Locking-force cycle; vibration; safety-stop test |
| 14 | Deck (EC) | Flexural strength FE-validated; impact-energy test |
| 15 | Handgrip + throttle (EE) | Cycle-test (≥ 100 k operations); LED contrast (ISO 9241-303) |
| 16 | Wheel assembly (EG) | Static + dynamic load; spoke tension; runout |
| 17 | Fastener + bolted joint (DT) | Torque-tension test (axial + transverse); vibration loosening; salt-spray |
| 18 | Thermal management (DV) | Pull-down chamber; IR thermography; NTC sensor calibration |
| 19 | EMC/EMI (DX) | Anechoic chamber (CISPR 32; UNECE R10) for emissions + immunity |
| 20 | Cybersecurity (DZ) | Penetration test; fuzzing of BLE/OBD/CAN; secure-boot bypass attempts; TARA scenarios |
| 21 | NVH (EB) | Sound-power semi-anechoic chamber (ISO 3744); accelerometer-array; FFT |
| 22 | Functional safety (ED) | HARA evidence + ASIL-graduated tests; HiL; back-to-back simulation |
| 23 | Sustainability (EF) | LCA gate-to-gate inventory; recyclability fraction; battery-passport disclosure |
| 24 | Repair (EH) | Disassembly-time test; spare-parts availability; documentation completeness |
| 25 | Environmental robustness (EJ) | IEC 60068-2 combined cycle (humidity + temperature + vibration); salt-spray (ISO 9227) |
| 26 | Privacy (EL) | DPIA per GDPR Art. 35; data-flow validation; pen-test of PII exposure; cookie/consent audit |
| 27 | Reliability prediction (EN) | MTBF/MTTF empirical demonstration test; HALT; HASS; accelerated-life Arrhenius |
| 28 | SW & firmware (EP) | Unit tests + integration tests + V&V activities per IEC 61508-3 |
| 29 | Human factors (ER) | Usability test (formative + summative per ISO 9241-11); SAGAT/NASA-TLX measurements |
| 30 | Manufacturing quality (ET) | PPAP submission package; SPC capability demonstration Cpk ≥ 1.33; Gage R&R |
| 31 | Risk management (EV) | Risk-register treatment-effectiveness V&V; LOPA IPL functional test; ALARP demonstration evidence |
| 32 | Regulatory (EU type approval, FCC, UNECE) | Type-approval test in an accredited 3rd-party lab per ISO/IEC 17025 |
V&V is the methodology that covers all of the above with a single vocabulary (IEEE 1012 + ISO/IEC/IEEE 29119) + a single process model (V-Model/W-Model + 12207/15288 V&V processes) + a single documentation set (IEEE 1028 review records + 29119-3 test docs + RTM).
16. Owner-level V&V “tells” — DIY checklist
A consumer will not see lab reports or an RTM. But there are 8 proxy-tells that correlate with V&V depth:
- Test reports availability — does the manufacturer publish test reports for type approval / FCC ID / UNECE R10 EMC? Public test-report databases (e.g., FCC OET for radio devices) are a strong V&V signal. A vague claim “certified” without a test-report document is weak.
- Certification body — TÜV Rheinland / TÜV SÜD / Intertek / UL / SGS / DEKRA / Bureau Veritas mark on the product + on documentation. Tier-1 TIC partner = robust IV&V; an unknown “cert ABC123” = weak.
- Independent test-lab marks — UL Listed / UL Recognized / ETL / CE NoBo / FCC ID — third-party V&V evidence.
- Manufacturer field-issue track-record — recall history (NHTSA + EU RAPEX/Safety Gate + UK PSD) — a manufacturer that recalls + transparently communicates field issues has an effective V&V loop (their IV&V detects + fixes; vs a manufacturer with public field failures + no recall = weak V&V).
- Datasheet spec ↔ measurement traceability — if the datasheet claims “brake stopping distance 4 m at 20 km/h on dry pavement”, look for an accompanying test report citing the test method + accredited lab + sample size + measurement uncertainty. A bare claim without traceable evidence = weak V&V.
- Firmware-update CVE-disclosure record — a manufacturer publicly tracking + patching CVEs (security V&V) — strong IV&V (penetration testing). No CVE record + no firmware updates = weak.
- Long-term warranty depth — a manufacturer offering ≥ 2 yr warranty with explicit failure-mode coverage = confidence in reliability V&V; 6 months / 1 yr only = weak V&V or weak confidence.
- Documentation completeness — a full owner manual with safety warnings + maintenance procedures + spare-parts list + technical specifications + test-certificate references — evidence that the QA process is mature; a minimal manual = weak SQA + likely weak V&V.
Yellow flags: “certified” without a named certification body; “tested” without a test method or lab name; a spec sheet with dozens of numbers but no measurement-uncertainty disclosure; a vague firmware-update changelog (“bug fixes”); no recall history but no transparent issue list either.
Green flags: published test reports with an accredited lab + sample size + uncertainty; recall transparency with RCA detail; a warranty with documented MTTF/MTBF; CVE-disclosure timeline; certification marks UL/TÜV/etc.; a comprehensive owner manual + Hazardous Substances declaration + WEEE marking + chemical-disclosure per REACH/RoHS.
17. Future axes — where the axis series will extend
Visible future axes:
- Production logistics + supply-chain security — ISO 28000:2022 Security and resilience — Security management systems; C-TPAT (Customs-Trade Partnership Against Terrorism); AEO (Authorized Economic Operator); UFLPA (Uyghur Forced Labor Prevention Act US 2021); EU Forced Labor Regulation 2024.
- Configuration management — ISO 10007:2017 Quality management — Guidelines for configuration management; IEEE 828:2012 software configuration management; CMII (Configuration Management II); baseline + change-control + status accounting + audit. The foundation for managing HW + SW + firmware versions.
- Project management — ISO 21500:2021 Project, programme and portfolio management — Context and concepts; PMBOK 7th ed. 2021 (PMI); PRINCE2 6th ed. 2017 (Axelos); Agile + Scrum (PMI-DASSM, Scrum.org); IPMA-ICB4.
- Sustainability impact assessment — ISO 14040:2006 + ISO 14044:2006 LCA Life-cycle assessment — Principles + framework + Requirements; ISO 14025:2006 Type III environmental declarations (EPD); ILCD International Reference Life Cycle Data System (EU JRC); product carbon footprint ISO 14067:2018; integration with the EU Ecodesign for Sustainable Products Regulation (ESPR) 2024.
- Maintenance engineering — EN 13306:2017 Maintenance terminology; EN 17007:2017 Maintenance process and associated indicators; IEC 60300-3-14:2004 Maintenance and maintenance support; RCM-II (Moubray) + RCMa MIL-STD-3034 + MSG-3 aerospace.
Each of those will add a new axis and potentially a new process meta-axis following the same structure (standard deep-dive + worked e-scooter example + cross-axis matrix + DIY checklist).
Recap — V&V concept-as-pattern
What this V&V engineering axis covered:
- V&V — verification (“Are we building the product right?” — Boehm 1979) + validation (“Are we building the right product?”). Verification is an internal-consistency check against spec; validation is an external-fitness check against user need.
- IEEE 1012:2016 — the core V&V standard: aligned with ISO/IEC/IEEE 15288:2015 (system) + 12207:2017 (software); V&V tasks per life-cycle stage; 4 integrity levels (1-4) with risk-graduated rigor; IV&V with 3 independencies.
- ISO/IEC/IEEE 29119 family — a five-part testing standard: Part 1:2022 vocabulary; Part 2:2021 processes; Part 3:2021 documentation (replaces IEEE 829); Part 4:2021 techniques (specification/structure/experience-based); Part 5:2024 keyword-driven.
- Coverage criteria — statement < branch < decision < condition < MC/DC < path; MC/DC is mandatory for DO-178C Level A + ISO 26262 ASIL D.
- Reviews + inspections — IEEE 1028:2008 + Fagan IBM 1976: 60-90% of defects detected by formal inspection vs 20-50% by testing.
- V-Model + W-Model — life-cycle visualisation: V-Model decomposition + verification mirror; W-Model parallel V&V + shift-left testing.
- RTM (traceability matrix) — requirements → design → code → tests, bidirectional. Required by IEEE 1012 + ISO 26262 + DO-178C.
- Risk-based testing + mutation testing + regression — RBT prioritises V&V effort by risk-management EV output; mutation testing tests the tests; regression catches change-induced breaks.
- ISO 26262-8:2018 + DO-178C — cross-industry V&V for high-assurance: ASIL coverage requirements; DO-178C 5 DALs A-E with graded objectives.
- Cross-axis matrix — a V&V activity for each of the 32 prior axes; converts a spec claim into engineering evidence.
V&V engineering is the verification-validation meta-axis of the entire e-scooter. Without it, specs for reliability MTTF, ALARP risk tolerance, ASIL C controllability, GDPR DPIA conclusion, Cpk ≥ 1.67 manufacturing capability, ANSUR P5-P95 ergonomic fit, IP67 and all the other 32-axis claims remain paper claims — not engineering evidence. V&V is the methodology that converts each prior axis from claim into evidence. The owner-as-buyer will not see lab reports + RTM, but 8 proxy-tells (test-report availability, certification body, independent-lab marks, recall transparency, datasheet ↔ measurement traceability, CVE record, warranty depth, documentation completeness) correlate with V&V depth and let one differentiate a manufacturer with a robust V&V process from one with paper claims.