The Seven Testing Principles

Why it matters

Teams that ignore the principles make predictable mistakes: they promise "bug-free" releases, test everything to the same depth and run out of time, start testing only when the code is finished, and trust a regression suite that stopped finding anything long ago. Each mistake costs money or trust when a failure reaches users.

For you as a tester, the principles are the arguments behind your decisions: why the payment module gets more testing than the settings page, why you want to see a story before it is built, and why "all tests passed" is never the whole report. A decision is easier to accept when you can name its reason.

The seven principles in ISTQB CTFL v4.0

Section 1.3 of the ISTQB Foundation Level syllabus v4.0 lists these seven principles, under these names. Each sums up long experience across many projects, and each changes something you do.

• 1. Testing shows the presence, not the absence of defects: it can show that defects exist, but cannot prove that there are none. It lowers the chance that defects remain undiscovered.
• 2. Exhaustive testing is impossible: testing everything is not feasible except in trivial cases, so test techniques, test case prioritisation and risk-based testing are used to focus the effort.
• 3. Early testing saves time and money: a defect removed early causes no further defects in the work products built on it, and fewer failures occur later in the lifecycle.
• 4. Defects cluster together: a small number of components usually contain most of the defects found, or cause most failures in operation, an illustration of the Pareto principle.
• 5. Tests wear out (older name: the pesticide paradox): tests repeated many times without change find fewer and fewer new defects.
• 6. Testing is context dependent: no single approach fits every product; testing is done differently in different contexts.
• 7. Absence-of-defects fallacy (older name: absence-of-errors fallacy): verification alone does not make a system successful; it must also meet its users' needs, so validation is needed as well.

What testing can and cannot claim: principles 1, 2 and 7

Principle 1 changes how you report. A bank's app passes all 250 regression tests; a week after release, customers find that standing transfers set for the 31st are skipped in months with fewer than 31 days. No test used that date. As in the lesson on why software testing matters, a passing run is evidence about what was tested and none about what was not, so report the scope, the results, what was not tested and the remaining risk, never "bug-free".

Principle 2 changes how you choose tests. The hotel search form below has five fields and over 17 million valid combinations, before invalid values, browsers and devices. So you select tests with test design techniques you will learn later in the roadmap: equivalence partitions and boundary values per field, pairwise combinations across fields, and the most depth where a failure would hurt most.

Principle 7 changes what you check besides the specification. A university's course registration portal meets all 40 specified rules and every defect found is fixed, yet students keep registering for courses whose lectures overlap, because the portal never shows a timetable. As in the lesson on verification and validation: watch real users try the main task, join acceptance testing, and report such gaps even when no requirement is broken.

Hotel search: every valid combination

city          50
check-in     365   (dates up to one year ahead)
nights        30   (1..30)
guests         8   (1..8)
room type      4
------------------------------------------------
50 x 365 x 30 x 8 x 4 = 17,520,000 combinations
At one test per second: about 203 days, nonstop

Try it: Pairwise Testing Generator

Where and when to put the effort: principles 3 to 6

Principle 3 changes when you start. The lesson on why software testing matters followed one requirement defect to three moments, and each later moment cost more; the same holds for every story. A story says "Users can upload a profile photo of up to 5 MB." In refinement you ask which formats are allowed, whether 5 MB means 5,000,000 or 5,242,880 bytes, and what users see when a file is too large. Three answers now can prevent three defects later. Review stories, acceptance criteria and designs as soon as they exist, and run tests as soon as something can run.

Principle 4 changes where you look hardest. In a shop's last three releases, most logged defects were in the promotions module, where discount rules combine. That module gets more test cases, an exploratory session and a close look at every change, and the cluster becomes an input to risk-based testing. Keep lighter testing everywhere else: a new feature or a rewrite can start a new cluster.

Principle 5 changes how you maintain tests. A checkout suite has used the same three products and one test card for a year; it stays green while a defect in products with size variants reaches customers. Vary the test data and add tests for new features, changed code and every escaped defect. Repetition is still right for automated regression tests, whose job is to show that what worked still works.

Principle 6 changes how formal and deep you go. An insulin dose calculator needs systematic test design, traceability to requirements, documented evidence and independent review. A two-week marketing landing page needs a quick check of content, links, forms and layout on its visitors' browsers. Before choosing an approach, ask about the domain, risks, regulations, the team's lifecycle and the deadline.

Try it: Acceptance Criteria Generator Try it: Risk-Based Test Generator Try it: Regression Scenario Generator

Worked example: "Can you confirm there are no bugs by Thursday?"

A pharmacy delivery app adds prescription upload: the customer photographs a prescription and a pharmacist checks it before the order is prepared. On Monday the delivery manager asks: "Can you test everything and confirm there are no bugs before Thursday's release?" The tester answers with the principles, turning the request into a plan instead of a yes or a no:

• Principle 1: "I can't confirm there are no bugs. On Thursday I can show what I tested, what failed, what I could not test and the risk that remains."
• Principle 2: "Every phone, file type, photo quality and prescription state together is far too many cases, so I will pick the riskiest combinations and cover the rest pairwise."
• Principle 4: "Most of our defects in the last two releases were in prescription checks, so that part gets the deepest testing."
• Principle 5: "Our regression suite has not changed since spring; I will add cases for expired prescriptions and blurred photos, which it never covered."
• Principle 6: "This is health data and prescription medicine, so I will keep evidence traced to each requirement, more formally than for a promotions banner."
• Principle 7: "Can a pharmacist try the check screen on Tuesday? It could meet the specification and still be too slow at a busy counter."
• Principle 3: "For the next feature, invite me to refinement, so questions such as which file types we accept are answered before coding."

Frequently asked questions

References

• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• ISTQB GlossaryISTQBThe standard definitions of testing terms — test case, regression testing, severity, priority, boundary value analysis and the rest.
• Lessons Learned in Software Testing: A Context-Driven Approach (Cem Kaner, James Bach, Bret Pettichord)Wiley · 2001Practical lessons on bug reporting, test design, automation and managing testing — the context-driven school.