Test Management and Tools

Why it matters

Without a shared record, test management breaks down as soon as a team has several testers or a release has several builds. Results sit in personal spreadsheets, nobody can say which build a "pass" belongs to, status reports take hours to assemble and are out of date when sent, and the release meeting argues about numbers instead of risks.

A tool does not repair a weak process, but a well-set-up one makes the process visible: progress, coverage and blockers can be read at any time, history survives people leaving, and an auditor can see who ran what on which build. A badly set-up one can be worse than a spreadsheet, because stale cases and unmapped results produce confident dashboards that are wrong. Knowing how the tool computes its answers is part of the tester's job.

Test management activities and what a tool records for each

In ISTQB terms test management is a role rather than a job title: it takes overall responsibility for the test process and focuses on three activities that frame analysis, design, implementation and execution. ISO/IEC/IEEE 29119-2 groups the same three as its test management processes. In agile teams the team itself handles some of these tasks.

Two supporting disciplines serve all three. Configuration management identifies and versions testware and test items, so every result names its exact build and environment and every test case has a history; an item approved for testing becomes a baseline that changes only through change control. Defect management moves each reported anomaly through an agreed workflow from discovery to closure. Each activity, and what a tool records for it:

• Test planning: set the objectives, scope, approach, entry and exit criteria, resources and schedule. In a tool: a plan per release, the suites and requirements in scope, cycles and assignments.
• Test monitoring: gather information continuously to compare progress with the plan and check the exit criteria. In a tool: live counts of results, coverage per requirement and linked defects.
• Test control: act on what monitoring shows. The syllabus examples are reprioritising tests when a risk becomes an issue, re-checking entry or exit criteria after rework, adjusting the schedule when an environment arrives late and adding resources when and where needed. In a tool: changed priorities, reassigned tests, a new run for a new build.
• Test completion: at a milestone such as a release or the end of an iteration, consolidate results, archive or hand over useful testware, record lessons learned and write the test completion report. In a tool: a closed cycle, the final report export, open defects carried forward.

What a test management tool does, whatever the product

The syllabus lists management tools among the kinds of tool support for testing: they make the test process more efficient by helping to manage the SDLC, requirements, tests, defects and configuration. It also counts a spreadsheet as a test tool. Products differ in names and screens, and this lesson recommends none of them; look for capabilities, not brands.

Choose from your process, not from a feature list: write down how the team plans, runs, links and reports, list the reports stakeholders really need, and try a short list of tools on one real release. The capabilities to look for:

• Test repository: test cases with ids, preconditions, steps, expected results, priority and owner, grouped into suites, with shared steps, parameters and a version history.
• Planning and execution: plans or cycles per release, runs per build and environment, assignment to testers, and step-by-step results with evidence attached.
• Integrations: requirements and defects usually stay in the team's issue tracker and are linked, not copied; automated results arrive from CI, commonly as a JUnit XML report, and are mapped to test cases.
• Traceability and reporting: links between requirements, tests, results and defects, and the progress, coverage and defect reports computed from them.
• Administration: roles and permissions, an audit trail of who changed or executed what, custom fields and review workflows, import, export and an API.

Try it: XML Formatter Try it: JSON to CSV

Traceability and reporting in a tool

In the requirements traceability lesson the links lived in a matrix. In a tool each link is a field on a record: a test case names the story or requirement it covers, a result belongs to a run that names its build and environment, and a defect is linked to the failed result. The matrix becomes a report the tool generates, current the moment a result is recorded.

Every report is a query over those records, so learn the rules behind a report before you present it, and agree them with the team. The counting rules from the execution reporting lesson still apply:

• Scope: which release, build and environment. A pass on last week's build says little about today's.
• Latest result: when a test ran several times in scope, the latest result usually counts; a fail followed by a pass on the same build is a question to investigate, not a pass.
• Requirement status rule: for example, passed only when every linked test passed in scope and no linked defect of severity Major or higher is open; not covered when nothing is linked.
• Mapping: automated results count only when they are mapped to test cases; unmapped results are invisible to coverage.
• Export: keep the data behind a dashboard, for example as CSV, for the completion report and audits.

Try it: Requirements → Test Cases Try it: CSV to PDF

Worked example: reading a release coverage report

Sahl Clinics (fictional) is an appointment booking app. Release 4.2 has five user stories and 38 linked test cases. Most are run manually on staging; the API tests are automated, and their JUnit XML results are imported from CI and mapped to their test cases. On the morning of the go/no-go meeting the lead exports the coverage report for the release candidate:

• The headline pass rate is 97% (33 passed of 34 executed), yet only BK-101 is ready. Read by story, not by total.
• BK-104 has no linked tests. Either tests exist but were never linked (search the repository and any unmapped CI results) or the SMS reminder was never tested. Only coverage computed from links reveals this.
• BK-102 failed and D-311 is open: triage and the product owner decide whether to fix and confirm on the next build or to release with an accepted, recorded risk.
• BK-103 is blocked by D-308. Blocked means not tested: an unknown risk, never a pass.
• BK-105 has one test not run: run it today or record why not.
• The control actions go into the tool as well: new assignments and a run for rc4 from the same suite, so the rc3 results stay the record. The final export becomes an appendix of the test completion report.

Requirements coverage: release 4.2, build 4.2.0-rc3, staging
Counting: latest result of each test in this scope
Story status: Passed = every linked test passed and no linked defect
              of severity Major or higher is open

Story   Summary              Tests  Pass  Fail  Blocked  Not run  Defects  Status
BK-101  Book an appointment  12     12    0     0        0        -        Passed
BK-102  Free cancellation    8      6     1     0        1        D-311    Failed
BK-103  Reschedule           6      4     0     2        0        D-308    Blocked
BK-104  SMS reminder         0      -     -     -        -        -        Not covered
BK-105  Staff calendar view  12     11    0     0        1        -        In progress
Total                        38     33    1     2        2

Try it: Percentage Calculator

Frequently asked questions

References

• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• ISO/IEC/IEEE 29119-2:2021 — Software and systems engineering — Software testing — Part 2: Test processesISO/IEC/IEEE · 2021The test process model: organisational, test management and dynamic test processes, with their activities and outcomes.
• Certified Tester Advanced Level Test Management (CTAL-TM) v3.0ISTQB · 2024ISTQB's advanced syllabus on managing testing: test planning and estimation, risk-based testing, monitoring, metrics and reporting.