Test Case Quality and Review

Why it matters

A test case is run many times, and its result is what the team reports. A case with a vague expected result passes whatever the product does, so a real defect can reach production while the report says the requirement was tested. A missing precondition does the opposite: the case fails on a correct product and a developer chases a bug that does not exist.

Before execution, the fix is a few edited lines. Your test cases are also the visible record of your work: new team members learn the product from them and automation engineers turn them into scripts.

What makes a test case good

The ISTQB glossary describes a test case as preconditions, inputs, actions, expected results and postconditions, developed from test conditions. Any of these parts can be wrong, missing or unclear. There is no single standard list of quality attributes, but most review checklists come down to these:

• Traceable: it names the requirement, acceptance criterion or risk it covers.
• Correct: its expected result comes from the test basis, not from what the system does today.
• Unambiguous: two testers would perform the same actions and judge the result alike.
• Verifiable: each expected result names something observable, such as a value, a message or a status.
• Focused: it checks one behaviour, so a failure points at one cause.
• Independent and repeatable: it states or creates what it needs and gives the same result in any order.
• Maintainable, at the agreed level of detail: exact where the team needs exact values, with shared steps referenced rather than copied.

A test case review checklist

Review cases before their first run, before they join the regression suite, and when a requirement they trace to changes. For most cases an informal peer review is enough: another tester reads them with the story open.

Keep the checklist short and grow it from the defects you actually find. A starting point:

• Basis: does the case name what it covers, and does that item still say what the case assumes?
• Title: does it state one behaviour and its condition?
• Preconditions: is every required state written down, such as role, record status and settings?
• Test data: are exact, synthetic values given, or a named data set?
• Steps: is each step a clear action that a new team member could follow exactly as written?
• Expected results: is each one observable, specific and taken from the test basis?
• Focus and independence: does the case check one behaviour and run correctly on its own, in any order?
• The set: is every acceptance criterion covered, are negative and boundary cases present, and are there duplicates?

Try it: Requirements → Test Cases Try it: User Story → Test Cases

Defects that reviews find in test cases

Test cases have defects like any other work product. Each type below makes a case pass when it should fail, fail when it should pass, or cost time without adding coverage.

• Vague expected result ("works correctly", "an error is shown"): the case passes whatever happens.
• Expected result copied from the system: if that behaviour was wrong, the case now confirms the defect.
• Missing precondition or data: the case fails for reasons unrelated to the product, or passes only with the author’s own data.
• Several behaviours in one case: a failure does not show which behaviour broke.
• Hidden dependency: the case works only after another case has created its data.
• Outdated case: the requirement changed and the case did not.
• Duplicates: two cases test the same condition under different titles.
• Gaps in the set: an acceptance criterion, negative path or role with no case at all.

Try it: AI Duplicate Test Case Detector

Worked example: reviewing an order cancellation case

Story ORD-31 lets a customer cancel an order before it ships. AC1: an order with status Placed or Paid can be cancelled from My orders after a confirmation. AC2: a cancelled order shows status Cancelled and a full refund to the original payment method starts. AC3: a Shipped order shows no Cancel order button.

A colleague wrote TC-ORD-07. You review it with the story open and mark each finding major (the case could give a wrong result) or minor (it is harder to read or maintain).

Part 1:1ID:              TC-ORD-07
Part 2:2Title:           Order cancellation
Part 3:3Requirement:     -
Part 4:4Preconditions:   User is logged in
Part 5:5Test data:       Any order
Part 6:6Steps:
  1. Open My orders and select the order
  2. Click "Cancel order" and confirm
Part 7:7  3. Check the refund and that a shipped order cannot be cancelled
Part 8:8Expected result:
  Order is cancelled correctly and the refund works

• Passes: the ID, and steps 1 and 2, clear actions a new team member could follow. The rework only names the dialog button.
• Title, minor: it names a feature, not a behaviour. Suggested: "Paid order can be cancelled before it ships".
• Requirement, major: empty, although the case covers ORD-31 AC1 and AC2.
• Preconditions and test data, major: "Any order" may be a shipped one, which fails on a correct product, and gives no refund amount to check.
• Step 3, major: it checks AC2 and AC3 together, though AC3 needs a shipped order and its own case.
• Expected result, major: "correctly" and "works" cannot be observed, so the case cannot fail.
• The set, major: no case covers AC3, or cancelling a Placed order that is not yet paid.

ID:              TC-ORD-07
Title:           Order cancellation
Requirement:     -
Preconditions:   User is logged in
Test data:       Any order
Steps:
  1. Open My orders and select the order
  2. Click "Cancel order" and confirm
  3. Check the refund and that a shipped order cannot be cancelled
Expected result:
  Order is cancelled correctly and the refund works

Recording findings and checking the rework

Log each finding where the author will see it: case ID, checklist item, what is wrong and a suggested fix. Comment on the case, not the person. Some findings are gaps in the requirement: AC2 promises a refund for every cancelled order, but a Placed order was never paid. Ask the product owner instead of guessing in the test case.

Then check the rework by comparing the old and new versions line by line. A common exit rule is no open major findings. The reworked case and the cases the review added:

ID:              TC-ORD-07
Title:           Paid order can be cancelled before it ships
Requirement:     ORD-31 AC1, AC2
Preconditions:   Customer sara.k@example.com is signed in
                 Her order ORD-1042 has status Paid
Test data:       ORD-1042: 349.00 SAR, paid by card
Steps:
  1. Open My orders and select ORD-1042
  2. Click "Cancel order"
  3. In the dialog, click "Yes, cancel"
Expected result:
  - ORD-1042 shows status "Cancelled" in My orders
  - The "Cancel order" button is no longer shown
  - Order details show a refund of 349.00 SAR to the card, status "Started"

Added by the review:
  TC-ORD-08  Shipped order shows no "Cancel order" button       (ORD-31 AC3)
  TC-ORD-09  Placed order that is not yet paid can be cancelled  (ORD-31 AC1)

Try it: Text Diff Try it: AI Test Case Optimizer

Frequently asked questions

References

• ISTQB Certified Tester Foundation Level (CTFL) v4.0ISTQB · 2023The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
• ISTQB GlossaryISTQBThe standard definitions of testing terms — test case, regression testing, severity, priority, boundary value analysis and the rest.
• ISO/IEC 20246:2017 — Software and systems engineering — Work product reviewsISO/IEC · 2017A generic framework for reviewing work products — the review process, roles and review techniques.
• ISO/IEC/IEEE 29119-3:2021 — Software and systems engineering — Software testing — Part 3: Test documentationISO/IEC/IEEE · 2021Templates and content for test documentation — test policy, strategy, plan, status and completion reports, incident reports.