Test Completion and Closure

Why it matters

Without completion, much of what testing produced is lost. Unfixed defects stay "open" with nobody responsible until a customer reports them. Test data and scripts sit in personal folders, so the next release rebuilds its regression tests from scratch. Temporary accounts and copies of personal data stay in test environments, which is a security and privacy risk.

For the tester, completion makes the work visible and reusable. A clear report shows what was tested, what was not and which risks remain, so stakeholders decide on the release knowing what they accept. Lessons learned that become actions with owners let the team find problems earlier next time.

When test completion happens

CTFL v4.0 places test completion at project milestones: a release, the end of an iteration, or the completion of a test level such as system testing. Agile teams usually do a light completion every sprint and a fuller one for each release; a hotfix needs only a small one.

The usual trigger is that the exit criteria in the test plan have been evaluated. Testing can also stop earlier: when time or budget runs out, CTFL v4.0 accepts ending testing if the stakeholders have reviewed and accepted the risk of going live without further testing, and a project can be cancelled. Completion then matters even more, because the report must say plainly what was not done. A temporary pause is different: that is a suspension of testing, not its completion.

The five completion tasks

CTFL v4.0 lists five tasks for test completion (older syllabi called them test closure activities), and ISO/IEC/IEEE 29119-2 describes a test completion process with the same core: archive the test assets, clean up the test environment, identify lessons learned and report. Each task answers one question.

• Unresolved defects — what happens to them? Each one gets an owner and a recorded decision, usually from the product owner: deferred to a later release, which makes it a change request or a product backlog item and puts it on the known-issues list; closed as won't fix, a real defect the business accepts; or rejected, because it turned out not to be a defect.
• Testware — what is worth keeping? Test cases, automated tests, test data and its scripts, environment configuration and session notes that will be useful later are archived or handed over; the tests of removed features are retired.
• Test environment — what state do we leave it in? Shut it down or reset it to the state agreed with its owners: a known baseline, kept for hotfix testing, or released to another team. Remove temporary accounts and access, and delete copies of personal data.
• Lessons learned — what do we change next time? Analyse what slowed testing, which defects were found late or escaped to production, and how the actual effort compared with the estimate. Each finding becomes an improvement action with an owner.
• Test completion report — what do stakeholders need to know? Testing against the plan, deviations, defects not fixed, unmitigated risks, metrics and lessons learned, sent to the stakeholders. The lesson on test reporting covers its contents; completion is when it is written.

Try it: Markdown to PDF

Handover to operations, support and maintenance

CTFL v4.0 says that testware may be handed over to the appropriate teams. After a release these are usually operations, support and whoever maintains the product. A handover is done when the receiving team can use what it gets without asking the tester what it means, so agree it with them before the release.

• Support: the known-issues list — each open defect in plain words, who is affected, the workaround and the backlog reference.
• Operations: the post-deployment smoke checks and the signals to watch after the release, such as the payment failure rate.
• Maintenance: the regression suite with the new tests and a test for every escaped defect, the test data scripts and the traceability from requirements to tests.
• Everyone: the completion report and where the archived testware is stored.

Try it: Regression Scenario Generator Try it: Requirements → Test Cases

Worked example: closing release 3.4 of a checkout

A food-delivery app releases version 3.4: wallet payments are added, a new coupon screen replaces the old one and cash on delivery is removed. Exit criteria: every high-risk requirement tested, no open Blocker or Critical defect, the regression suite passing. 186 of 192 planned test cases were executed; the product owner descoped 6 low-risk wallet cases because the wallet provider's sandbox was down for two days, so the high-risk criterion was still met.

Note two details in the record below: the 6 descoped cases appear as a deviation with their residual risk, which the product owner accepted in writing, and every line has an owner or a destination.

Test completion record: release 3.4 (checkout)

Trigger: go decision after the exit criteria were evaluated
Execution: 186 of 192 test cases; 6 low-risk wallet cases descoped
  (wallet sandbox down for 2 days), residual risk accepted in writing
  by the product owner
Defects:
- DEF-481 (Major) Discount shown twice after the coupon is re-applied;
  moved to PB-902, workaround: remove the coupon and apply it again
- DEF-487 (Minor) Receipt e-mail shows the date in the wrong format; moved to PB-903
Testware:
- 38 new API tests merged into the regression suite
- Test data script tagged release-3.4 and archived
- 6 cash-on-delivery tests retired (feature removed)
Environment: staging reset to baseline; 4 temporary admin accounts removed
Lessons learned:
- 2 days lost to the sandbox outage; action: book the sandbox in the test plan
  (owner: QA lead)
- DEF-455 escaped from release 3.3; action: regression test added (owner: tester)
Report: sent to the product owner, development lead and support lead
Handover:
- Support: known issues DEF-481 and DEF-487 with the workaround
- Operations: smoke checks; watch the payment failure rate for 48 hours

Frequently asked questions

References

• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• ISO/IEC/IEEE 29119-2:2021 — Software and systems engineering — Software testing — Part 2: Test processesISO/IEC/IEEE · 2021The test process model: organisational, test management and dynamic test processes, with their activities and outcomes.
• ISO/IEC/IEEE 29119-3:2021 — Software and systems engineering — Software testing — Part 3: Test documentationISO/IEC/IEEE · 2021Templates and content for test documentation — test policy, strategy, plan, status and completion reports, incident reports.
• The 2020 Scrum GuideKen Schwaber and Jeff Sutherland · 2020The definition of Scrum: its accountabilities, events and artifacts, including the Definition of Done that testing helps meet.