Static Testing

What static testing is — beyond “not executing the code”

The ISTQB Foundation Level syllabus (CTFL v4.0) devotes its third chapter to static testing and divides it into two families: reviews, in which people examine a work product, and static analysis, in which tools evaluate it against formal rules. Both share the same objectives: to find defects directly in the work product, to evaluate its quality and build confidence in it, and to create a shared understanding between the people who wrote it and the people who will build, test and operate from it.

The distinction that matters most is defect versus failure. A defect is a flaw in a work product — a contradictory sentence, an unhandled null value, a missing error response in a contract. A failure is what a running system does wrong because of a defect. Dynamic testing observes failures, and then someone debugs backwards to the defect. Static testing lands on the defect directly, usually with its exact location. That is why a static finding is typically cheaper to fix: there is nothing to reproduce, nothing to diagnose, and often no code to rewrite yet.

Anything a person can read and understand can be reviewed, including documents that will never run; anything with a formal structure — code, a schema, a model, a configuration file — can also be analysed by a tool.

• It is not proofreading. Spelling fixes are a by-product; the target is defects that would otherwise become wrong behaviour, rework or risk.
• It is not a substitute for dynamic testing. It cannot show that the system is fast, that services integrate, or that a user can complete a task.
• It is not only a developer activity. Testers review requirements and test documents, product owners and designers review each other’s work, and operations engineers review deployment configuration.

Static testing vs dynamic testing

The two are complementary, not alternatives. The differences below build on the comparison in the ISTQB syllabus; most teams need both at every test level.

• What it finds — static: defects, directly in the work product. Dynamic: failures, from which defects are traced by debugging.
• When it can start — static: as soon as a work product exists, including before any code. Dynamic: once something executable exists.
• What it can examine — static: executable and non-executable work products (requirements, designs, test plans, contracts). Dynamic: running software only.
• Quality characteristics — static: those that do not depend on execution, such as maintainability, consistency and conformance to standards. Dynamic: those that do, such as performance efficiency, reliability under load and real integration behaviour.
• Defects found more easily — static: ambiguous or contradictory requirements, unreachable or duplicated code, undeclared or uninitialised variables, mismatched interface parameters, deviations from coding standards, some security weaknesses, acceptance criteria that no test covers. Dynamic: wrong outputs for particular data, timing and concurrency failures, environment and configuration problems, usability problems.
• Cost of a finding — static: usually lower, because the location is known and less has been built on top of the defect. Dynamic: higher, because it includes reproduction, diagnosis and rework of what was built on the defect.

What can be statically tested, and what it typically finds

For example, a user story says “the discount applies to orders over 500 SAR”. A reviewer asks whether an order of exactly 500.00 SAR qualifies — a static finding, in one sentence, before any code. If nobody asks, the developer chooses “greater than” or “greater than or equal”, and a boundary-value test weeks later may reveal that the choice differs from what finance expected. The same defect, found in two places, at very different cost. The list below shows what each kind of work product typically hides.

• Requirements and user stories — ambiguity, contradictions, omissions (error paths, permissions, limits), untestable statements, duplicated or conflicting business rules.
• Architecture and design — poor modularisation, single points of failure, inefficient data structures or database design, missing error handling between components, unclear ownership of data.
• API contracts and schemas — inconsistent naming, missing error responses, types that contradict their examples, breaking changes between versions, fields without constraints.
• Source code — undefined or unused variables, unreachable or duplicated code, excessive complexity, unsafe calls, query strings built from user input, violations of coding standards.
• Configuration and infrastructure-as-code — secrets committed in plain text, insecure defaults, wrong values for an environment, invalid YAML, a cron schedule that never fires.
• Test documents — a test plan with no exit criteria, test cases with vague expected results, acceptance criteria without any test, duplicated or obsolete cases.

Try it: AI Requirement Ambiguity Detector Try it: AI API Contract Risk Analyzer

Early defect detection and defect prevention

Defects tend to be introduced early and discovered late. A misunderstanding in a requirement becomes a design decision, then code, then tests written to the same misunderstanding — and every layer built on it has to be reworked when the problem is finally noticed. Static testing shortens that feedback loop: the requirement is questioned the day it is written, the design before it is coded, the change before it is merged. The ISTQB syllabus covers early and frequent stakeholder feedback together with the review process and notes that it can prevent misunderstandings about requirements.

Early detection and defect prevention are related but different. Detection finds this defect sooner; prevention stops the next one from being made. Reviews contribute to prevention in two ways. People learn: a developer whose changes keep drawing “input not validated” comments starts validating input. And findings feed back into the process as checklist items, templates, coding rules and lint rules. A defect type that appears in several reviews is a candidate for an automated check, so that no human has to look for it again.

• Classify review findings by type — ambiguity, missing error handling, naming, security, test gap. The counts show where defects come from.
• Fix the template, not only the document: a user-story template with an “error cases” heading removes a whole class of omissions.
• Bring testers into story refinement. Asking “how would I test this?” is itself a static test.

Reviews: the human side of static testing

A review is a structured examination of a work product by people other than, or in addition to, its author. Reviews range from a colleague reading a pull request to a formal inspection with trained roles, entry criteria and collected metrics. CTFL v4.0 describes four review types along that formality scale: the informal review, the walkthrough, the technical review and the inspection. IEEE 1028-2008, the IEEE standard for software reviews and audits (now listed by IEEE as inactive-reserved), additionally defines management reviews and audits, which assess progress and conformance rather than the technical content of a work product. ISO/IEC 20246 describes a generic framework for work product reviews.

Reviews draw on one generic process — planning, review initiation, individual review, communication and analysis, fixing and reporting — and the more formal the review, the more of it is performed; an informal review follows no defined process. The principal roles are manager, author, moderator (facilitator), scribe (recorder), reviewer and review leader. The page on software review types compares the four types on purpose, formality, leader, roles, documentation and metrics, and walks through the process, the roles and the success factors in depth.

• Informal review — a quick second look with no defined process.
• Walkthrough — the author leads participants through the work product to explain it and reach a common understanding.
• Technical review — qualified peers, led by a moderator, reach decisions on a technical question.
• Inspection — the most formal type, aimed at finding as many defects as possible, with metrics.

Requirements, design, code and test-document reviews

The review type describes how formal a review is; the work product decides what reviewers look for. The same team might run a walkthrough of a design, an informal review of most code changes and an inspection of the payment rules.

• Requirements reviews — check ambiguity, completeness, consistency, correctness, testability, feasibility, traceability and non-functional needs, ideally with product, development and testing together. This is where static testing usually pays off most, and it has its own page with a checklist and a worked example.
• Design reviews — check that the design meets the requirements, including the non-functional ones: how failures propagate between services, where data lives and who owns it, what happens under peak load, how the system will be tested and observed. Ask “what happens when this dependency is slow or unavailable?” of every arrow in the diagram.
• Code reviews — check correctness, tests, readability, security and consistency with the codebase. Google’s published code review guide sets a pragmatic standard: approve a change once it clearly improves the overall health of the code, even if it is not perfect, and mark optional remarks (for example with “Nit:”) so the author knows what blocks the merge. Let tools enforce formatting and style so that human attention goes to logic, design and tests.
• Test plan reviews — check scope and what is explicitly out of scope, the risks the plan addresses, the approach per test level, entry and exit criteria, environments and data, responsibilities and schedule. A plan without exit criteria cannot tell anyone when testing is finished.
• Test case reviews — check that each case traces to a requirement or acceptance criterion, states its preconditions and data, has an observable expected result, and that the set covers negative and boundary cases without duplicates. The anatomy of a good test case is covered in the guide on writing test cases.

Try it: AI Acceptance Criteria Reviewer Try it: AI Test Case Optimizer Try it: AI Duplicate Test Case Detector Try it: AI Test Coverage Analyzer

Static analysis: categories of tools and what they check

Static analysis is the tool-driven half of static testing: a program reads code, a model or a structured document and reports violations of rules. It needs a formal structure — a programming language, a schema, a specification format — and it scales in a way people cannot: a linter checks every line of every change, every time, in seconds.

The names below are well-known examples in each category, not a ranking. The right tool is the one that supports your languages, runs in your pipeline and produces few enough false alarms that the team keeps reading its output.

• Linters and style checkers — flag suspicious constructs and convention violations: unused variables, unreachable code, shadowed names, a missing await, inconsistent formatting. Examples: ESLint (JavaScript, TypeScript), Pylint and Ruff (Python), Checkstyle (Java), RuboCop (Ruby), ShellCheck (shell scripts), SQLFluff (SQL).
• Type checkers and compilers — check that values are used consistently with their declared types, catching whole classes of null and type errors before execution. Examples: the TypeScript compiler, mypy for Python, and the compilers of statically typed languages such as Java, C#, Go and Rust.
• Static application security testing (SAST) — trace data from untrusted sources to dangerous sinks and match known weakness patterns: SQL injection, cross-site scripting, path traversal, hard-coded secrets, weak cryptography. Examples: Semgrep, CodeQL, SonarQube, SpotBugs with security plugins. OWASP publishes guidance on what source-code analysis tools can and cannot do, and NIST’s SAMATE programme maintains test suites for evaluating them.
• Schema and specification validation — check that structured artefacts are valid against their specification: JSON against a JSON Schema, an OpenAPI document against the OpenAPI specification, XML for well-formedness, YAML and cron syntax. OpenAPI linters such as Spectral add consistency and style rules on top of structural validity.
• Complexity and maintainability metrics — cyclomatic complexity, function length, duplication and coupling, which point reviewers at code that is hard to test and to change.
• Dependency, secret and infrastructure scanners — check declared dependencies against vulnerability databases, search commits for credentials, and inspect infrastructure-as-code and container files for insecure settings. Examples: npm audit and pip-audit, OWASP Dependency-Check, gitleaks, hadolint, Checkov.

# Illustrative findings from three kinds of static analysis
src/checkout/discount.ts:41:18  error  "rate" is possibly "undefined".              TS18048 (type checker)
src/checkout/discount.ts:57:3   error  Unreachable code.                              no-unreachable (linter)
src/orders/search.ts:22:9       error  SQL query built by concatenating request
                                       parameter "q"; use a parameterised query.   sql-injection (SAST)
openapi.yaml  paths./orders/{id}.get.responses
                                warn   Operation defines no 4xx response.           (contract linter)

Try it: OpenAPI Validator Try it: JSON Schema Validator Try it: SQL Query Validator Try it: YAML Formatter

What static analysis cannot find

A clean static analysis report means the artefact passed the rules someone wrote. It says nothing about whether the software does what its users need.

Every tool also makes two kinds of error. False positives — warnings about code that is fine — erode trust until people stop reading the report; false negatives — defects the tool misses — create false confidence. A classic result of computability theory is that no analyser can decide every non-trivial behavioural property of arbitrary programs, so every tool trades one error against the other. Treat static analysis as a fast filter in front of reviews and dynamic tests, not as a verdict.

• Whether the behaviour is right. An analyser can confirm that a function returns a number; it cannot know the number should include VAT.
• Missing or misunderstood requirements. If the story never mentioned refunds, no tool will report that refunds are absent.
• Runtime and environment behaviour: response times, memory under real load, timeouts, integration with real services, configuration differences between staging and production.
• Most concurrency and timing defects, and defects that depend on particular data values reached only at runtime.
• Usability, and accessibility as experienced with a keyboard or a screen reader. Linters catch some markup problems, such as an image without alternative text, but not whether a page can actually be used.

Static testing in the CI/CD pipeline

Static checks are the cheapest stage of a pipeline, so they run first and on every change.

• In the editor and in pre-commit hooks — formatters, linters and type checks give feedback before a commit exists.
• On every pull request — lint, type check, SAST, schema and contract validation and secret scanning, with results shown inline on the diff so the human reviewer sees them.
• As a quality gate — new violations above an agreed severity block the merge; existing violations in legacy code are recorded as a baseline and paid down, instead of blocking everyone on the first day.
• Specification checks — validate OpenAPI, JSON Schema and configuration files in the same pipeline as the code that depends on them, so that a broken contract fails the build that introduced it.

# Illustrative stage order (commands are examples, not requirements)
stages:
  static-checks:
    - format-check        # formatter in check mode
    - lint                # e.g. eslint . --max-warnings=0
    - typecheck           # e.g. tsc --noEmit
    - validate-contracts  # OpenAPI and JSON Schema files
    - sast                # security rules on changed code
    - secret-scan
  unit-tests:          { needs: static-checks }
  integration-tests:   { needs: unit-tests }

A practical starting plan

• Add a formatter and a linter with a small agreed rule set; fail the build only on new violations.
• Validate every contract, schema and configuration file in the pipeline.
• Make testers part of story refinement and review every story against a requirements checklist before it enters a sprint.
• Agree a lightweight code review policy: who reviews, what blocks a merge, how long a change may wait for review.
• Reserve formal inspections for the few work products where a missed defect is expensive: payment and pricing rules, security-critical code, regulatory requirements, data migrations.

Try it: AI Requirement Gap Analyzer Try it: AI Acceptance Criteria Reviewer

Frequently asked questions

References

• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• ISO/IEC 20246:2017 — Software and systems engineering — Work product reviewsISO/IEC · 2017A generic framework for reviewing work products — the review process, roles and review techniques.
• IEEE 1028-2008 — IEEE Standard for Software Reviews and AuditsIEEE Standards Association · 2008The classic definitions of management reviews, technical reviews, inspections, walkthroughs and audits (listed by IEEE as inactive-reserved).
• ISTQB GlossaryISTQBThe standard definitions of testing terms — test case, regression testing, severity, priority, boundary value analysis and the rest.
• Source Code Analysis ToolsOWASP FoundationWhat static application security testing (SAST) tools do, their strengths and weaknesses, and how to choose one.
• Software Assurance Metrics And Tool Evaluation (SAMATE)NISTNIST's programme for evaluating software assurance tools, including static analysers, and the test suites used to measure them.
• Code Review Developer GuideGoogle (eng-practices)Google's published code review practices: what reviewers look for, the standard for approving a change and how to write review comments.