Entry and Exit Criteria

Why it matters

Starting without entry criteria means testing a build that is not testable: the login fails, an integration is missing, the data is wrong. Testers lose hours reporting failures caused by the environment, and nobody trusts the results. The ISTQB Foundation syllabus notes that an activity started without meeting its entry criteria is likely to be harder, slower, more costly and riskier.

Without exit criteria, "done" is decided by the deadline or by whoever argues longest. Criteria agreed in advance give the product owner facts for the release decision, and give the tester a neutral way to say "not yet".

Entry criteria: is the activity ready to start?

The ISTQB syllabus groups typical entry criteria into three kinds: available resources (people, tools, environments, test data, budget, time), available testware (test basis, testable requirements, user stories, test cases) and the initial quality of the test object, such as passed smoke tests. They should be defined for each test level and differ with its test objectives, so the entry to component testing is not the entry to acceptance testing.

If an entry criterion is not met, the team decides what to do: wait, start only the unaffected parts (API tests while the UI build is broken), or start anyway with the risk recorded and accepted by the people who agreed the criteria.

• Environment: the release candidate is deployed to the test environment and every service health check succeeds.
• Initial quality: the smoke suite passes 25 of 25 tests on that build.
• Testware and data: test cases for all High-risk requirements are peer-reviewed, and 50 test customer accounts are loaded, including Arabic names and expired cards.

Try it: API Health Checker

Exit criteria: is the activity complete?

The syllabus describes two kinds of exit criteria. Measures of thoroughness say how much was achieved: coverage reached, unresolved defects, defect density, failed test cases. Binary yes/no criteria say whether something was done: planned tests executed, static testing performed, every defect found reported, regression tests automated. The first edition of CTFL v4.0 called this second kind completion criteria; v4.0.1 renamed it so that it is not confused with exit criteria as a whole.

In sequential development models, the exit criteria of one test level are often part of the entry criteria of the next, as when system testing hands over to acceptance testing; in some iterative models this does not apply. Running out of time or budget can also end testing, but only when the stakeholders have reviewed and accepted the risk of going live without further testing. Record the unmet criteria and that acceptance.

Try it: Requirements → Test Cases Try it: Percentage Calculator

Suspension and resumption criteria

Suspension criteria name the situations in which continuing would waste effort or produce results nobody can trust; resumption criteria name what must be true before testing restarts. Suspending is a test control decision, like those in the lesson on test monitoring and control: monitoring shows the trigger, and control acts on it. Write the criteria in pairs:

• Suspend when the smoke suite fails on a new build; resume when a corrected build passes it.
• Suspend when the test environment or a required sandbox is down for more than 4 working hours; resume when it is restored and its health checks pass.
• Suspend when one defect blocks more than 30% of the remaining test cases; resume when its fix passes confirmation testing.
• On resumption, re-run the tests that were in progress and record the lost time in the next progress report.

Writing criteria that two people judge the same way

A criterion works only if two people looking at the same data reach the same yes or no. Give each one a measure, a threshold, a data source (test management tool, defect tracker, CI run) and the build it applies to. Criteria about High-risk test cases also need risk levels agreed before testing.

• "The build is stable" → "the smoke suite passes 25 of 25 tests on the build under test".
• "Testing is complete" → "all High-risk test cases passed and at least 95% of planned test cases executed".
• "No major bugs" → "no open Blocker or Critical defects; every open Major defect has an accepted workaround".
• "Good coverage" → "every in-scope requirement has at least one passed test case".
• "Performance is acceptable" → "with 200 concurrent users, 95% of searches respond in 2.0 seconds or less (p95)".

Try it: Risk-Based Test Generator

Worked example: checking exit criteria before a release

Release 3.5 of a food delivery app adds scheduled orders. Its system test plan has six exit criteria: all High-risk test cases executed and passed, at least 95% of the 150 planned test cases executed, no open Blocker or Critical defect, at most 3 open Major defects, the regression suite passing on the release candidate, and every in-scope requirement covered by a passed test. At the end of day 9 the test lead checks each one against the test management tool and the defect tracker.

• Met: 38 of 38 High-risk cases passed, 2 Major defects are open, the regression suite passed 212 of 212, and all 27 requirements have a passed test.
• Not met: 141 of 150 executed is 94%. The 9 cases left are Low-risk order-history checks, so the team runs them in one more day, or the product owner accepts the gap in writing.
• Not met: on day 8 an exploratory session found that changing the time slot of a scheduled order charges the card twice. The product owner will not accept a Critical defect that charges customers twice, so the fix comes first.
• The fix creates a new release candidate. The tester confirmation-tests the defect, re-runs the regression suite, re-checks all six criteria on that build and records the result in the test completion report.

Release 3.5, system testing: exit criteria at the end of day 9

Criterion                                   Target      Actual           Met
High-risk test cases executed and passed    100%        38/38 (100%)     yes
Planned test cases executed                 >= 95%      141/150 (94%)    no
Open Blocker or Critical defects            0           1 Critical       no
Open Major defects                          <= 3        2                yes
Regression suite on the release candidate   all pass    212/212          yes
In-scope requirements with a passed test    100%        27/27            yes

Frequently asked questions

References

• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• ISTQB GlossaryISTQBThe standard definitions of testing terms — test case, regression testing, severity, priority, boundary value analysis and the rest.
• ISO/IEC/IEEE 29119-3:2021 — Software and systems engineering — Software testing — Part 3: Test documentationISO/IEC/IEEE · 2021Templates and content for test documentation — test policy, strategy, plan, status and completion reports, incident reports.