White-Box Testing and Code Coverage

Why it matters

Black-box tests come from the requirements, so they cannot show how much of the implementation they reached. Code often holds more than the requirement says: error handling, special cases, old rules, a developer's shortcut. If no test runs that code, its defects reach production unseen.

For a tester, coverage turns "have we tested enough?" into concrete questions: which branches did no test reach, and does the requirement say what should happen there? Even testers who never write unit tests use uncovered code to find gaps in the test basis itself.

White-box and black-box: two sources of tests

The ISTQB glossary defines white-box testing as testing based on an analysis of the internal structure of the component or system. Its test basis is the implementation: code, but also architecture, workflows and data flows. Its coverage items, such as the statements or branches of a function, come from that structure, so the tests can be designed only once the design or the code exists.

Black-box techniques such as equivalence partitioning and boundary value analysis work from the specified behaviour, so they can be designed before any code exists and stay valid however it is written. The two views see different things:

• Strength: white-box testing considers the whole implementation, so it finds defects even when the specification is vague, outdated or incomplete.
• Weakness: a requirement that was never implemented leaves no code to cover, so the omission stays invisible. Black-box tests from the requirement reveal it.
• It works without running the code too: in a review or a dry run you trace inputs through the logic.
• It applies at every test level: branches in component testing, calls between components in integration testing, workflow steps in system testing.

How code coverage is measured

A coverage tool instruments the code (it adds counters, or asks the runtime to record execution), runs the tests and writes a report: per file, the share of statements or lines, branches and functions that ran, and the source with each line marked as run, not run or partly run. Line coverage is close to statement coverage but not identical: one line can hold several statements.

Criteria differ in strength: statement coverage asks whether every executable statement ran, branch coverage whether every branch ran, such as both the true and the false outcome of each if. The next lessons treat statement, branch, condition and path coverage in turn. Common tools:

• JavaScript and TypeScript: Istanbul (the nyc command, also behind Jest's --coverage by default) or the V8 engine's built-in coverage, which Jest and Vitest can also use.
• Python: coverage.py, which measures statements by default and branches with --branch.
• Java: JaCoCo, whose Java agent is added when the application starts, so it can also record coverage while a server is tested manually.
• In CI these tools publish the report and can fail the build below a threshold, such as Jest's coverageThreshold or coverage.py's --fail-under.

Designing tests from uncovered code

White-box design rarely starts from an empty page: teams design tests from the requirements, measure their coverage, then close the gaps. For each uncovered item:

• Read the decision that guards it: what must be true or false to reach it?
• Work back to an input: which parameter values, data or state make that condition hold? The comparison is itself a boundary, so boundary value analysis helps.
• Take the expected result from the test basis, never from the code: a test copied from the implementation confirms whatever the code does, defects included.
• If the requirement does not describe the behaviour, you have found a gap in the test basis: ask first.
• Rerun the tests: the item should now be covered and the new test should assert the result.
• If no input can reach the code, report possible dead code instead of forcing coverage.

Try it: Boundary Value Generator

Worked example: the line no requirement mentioned

Requirement: "Customers earn 1 loyalty point for every whole 10.00 SAR of an order. Gold members earn double points." Below are the developer's function, three black-box tests and the CI coverage report. Math.floor rounds down, so an order of 125.00 SAR earns 12 points.

Line 10 never ran. It gives 5 bonus points to app orders, a rule the requirement does not mention. Instead of copying +5 into an expected result, the tester asks the product owner, who confirms it: every app order earns 5 bonus points, including orders under 10.00 SAR.

• T4 loyaltyPoints(125, false, true) expects 17 and passes. Statement and branch coverage are now 100%.
• The confirmed rule also needs T5 loyaltyPoints(5, false, true) → 5. The code returns 0, because the early return on line 3 skips the bonus: a defect on a path that full branch coverage never required.
• The uncovered line exposed a missing requirement; the answer exposed a defect. The percentage showed neither.

 1  function loyaltyPoints(total: number, gold: boolean, app: boolean) {
 2    if (total < 10) {
 3      return 0;
 4    }
 5    let points = Math.floor(total / 10);
 6    if (gold) {
 7      points = points * 2;
 8    }
 9    if (app) {
10      points = points + 5;
11    }
12    return points;
13  }

Black-box tests                        Expected   Actual   Result
T1  loyaltyPoints(5, false, false)     0          0        passed
T2  loyaltyPoints(125, false, false)   12         12       passed
T3  loyaltyPoints(125, true, false)    24         24       passed

Coverage (CI run)   Statements    Branches    Not run
loyalty.ts          7/8  87.5%    5/6  83%    line 10: if (app) never true

Try it: Decision Table Generator

What coverage does not tell you

The lesson on test coverage covered the basics: executing code is not checking it, and code never written cannot appear as uncovered. When you design tests from coverage, also remember:

• Whether the expected results are right: a test can assert a wrong value, for example one copied from the code, and still count.
• Combinations: full branch coverage takes each decision both ways, not every combination. T1 to T4 reached 100% and missed "small order placed in the app".
• Data-dependent defects: a statement that ran with harmless data can fail with other values, such as a divisor of zero.
• Code outside the report: Jest, for example, counts only files the tests loaded unless collectCoverageFrom is set; libraries, database procedures, configuration and other services are usually not measured.
• Non-functional quality: performance, security, usability and accessibility need their own tests.

Try it: Requirements → Test Cases

Frequently asked questions

References

• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• ISTQB GlossaryISTQBThe standard definitions of testing terms — test case, regression testing, severity, priority, boundary value analysis and the rest.
• ISO/IEC/IEEE 29119-1:2022 — Software and systems engineering — Software testing — Part 1: General conceptsISO/IEC/IEEE · 2022The international software-testing standard: concepts, processes, documentation and techniques.
• NIST SP 500-235: Structured Testing: A Testing Methodology Using the Cyclomatic Complexity MetricNIST · 1996Structured (basis path) testing: McCabe's cyclomatic complexity used to derive a basis set of independent paths to test — more thorough than statement and branch coverage.