Execution Reporting
Execution reporting is how a tester tells the team what happened during a test run: how many tests have run, what they found, what could not be tested and what that means for the release. In everyday work it takes two forms: a short daily status at the end of each execution day, and a run summary when the run or cycle ends.
For example, "51 of 90 tests executed, 45 passed, 6 failed, 8 blocked" tells the team how far the run has come. "A Critical defect lets an order through with a rejected prescription, and all payment tests are blocked" tells them what is at risk, and that is the part people act on.
Why it matters
Release and fix decisions are made from these reports, often by people who never open the test management tool, so they cannot check the counts themselves. If attempts are counted as tests, or blocked tests as executed, the totals overstate progress and nobody can tell how much of the product has really been tested. Clear daily reporting turns problems into actions while there is still time to act.
For the tester, numbers that always add up and risks that prove real build credibility: people read your status first. And if a defect reaches production, the written record shows what was tested, what was not, and who accepted which risk.
Two reports: the daily status and the run summary
At the level of a single test run, testers write two small reports. The daily status goes out at the end of each execution day to the people who work with the run every day: the test lead, the developers and the product owner. It should be readable in a minute. The run summary is written once, when the run or cycle ends, and stays with the run as its record.
In ISTQB terms the daily status is a test progress report: it is produced while testing is under way so that test control can react in time, and ISO/IEC/IEEE 29119-3 calls such a report a test status report. The run summary covers one run, which is usually only part of a test level, so it feeds the next progress report and later the test completion report; on a small team it may itself serve as the test completion report for the cycle.
- Daily status: the run, the build and the time the data was taken; counts against the plan; new failed tests and blockers, each with an owner; what is at risk; what you need and from whom; the plan for tomorrow.
- Run summary: the final counts per area or risk level; the open defects behind the failures; what was not tested and why; the risks that remain; your assessment, labelled as yours.
Counting results so the numbers add up
The counts use the statuses from the lesson on test suites and runs. The ISTQB Foundation syllabus lists test cases run and not run, passed and failed among typical test progress metrics, and many test management tools add blocked. One definition keeps the counts honest: a test is executed when it ran to a verdict, passed or failed. A blocked test could not run, so it is neither executed nor failed: that part of the product is simply untested so far.
Count tests, not attempts. Some teams start a new run for every build, as in the lesson on test suites and runs; others keep one run open for the whole cycle and retest fixes on later builds inside it. Either way, each test counts once, with its latest result: a test that failed on Monday and passed after the fix on Wednesday is one passed test, and the failure stays in the defect report and the test history. Then check the arithmetic: passed, failed, blocked and not run, plus skipped if your run uses that status, must add up to the planned total.
- Always name the denominator: 86% of the executed tests passed, but only 60% of the whole run has passed so far.
- Skipped or descoped tests go on their own line with the reason and who decided, instead of disappearing from the plan so that progress looks better.
Run: 50 planned tests, end of day 2
Passed 30 + Failed 5 + Blocked 5 + Not run 10 = 50 planned (check)
Executed = Passed + Failed = 35
Progress = Executed / Planned = 35 / 50 = 70%
Pass rate = Passed / Executed = 30 / 35 = 86%
Passed of plan = Passed / Planned = 30 / 50 = 60%
Not executed = Blocked + Not run = 15 (30% of the run)Worked example: a five-day run, from daily status to summary
A pharmacy delivery app, release 1.8, has a five-day system test run: 90 planned tests in four areas, about 18 a day. The block below shows the progress of the run at 17:00 each day, taken from the test management tool, followed by the run summary written at the end of day 5.
On day 3 the tester's daily status read: "Day 3 of 5: 51 of 90 executed (plan 54); 45 passed, 6 failed, 8 blocked, 31 not run. Risk: Critical PH-231 lets an order go through with a rejected prescription. All 8 card and wallet payment tests blocked by the payment sandbox since day 2; owner: DevOps, no restore date. Need: a restore date or a decision on payment testing by day 4, 12:00. Tomorrow: order tracking and confirmation tests for fixed defects."
Each bar is the whole run of 90 planned tests, each test counted once with its latest result. Executed = passed + failed; blocked and not run are not executed.
- The pass rate rose from 88% to 95%, yet 16 tests (18%) were never executed. Card payments returned to the sandbox on day 5, but the 6 wallet payment tests stayed blocked, and 10 Medium and Low tests ran out of time.
- The failed count fell on days 4 and 5 because fixed defects passed their confirmation tests; each test counts once, with its latest result.
- The summary names the 3 defects behind the 4 failures and what was not tested. The tester's assessment, labelled as such: not ready while PH-231 is open; for wallet payments the product owner chooses between waiting for the sandbox and releasing with the wallet switched off.
TEST RUN: Pharmacy app 1.8 · system test, cycle 1 · builds 1.8.0-rc1 to rc3 on staging
Planned: 90 tests in 5 days (18 a day) · latest result per test · data at 17:00
Day Executed Passed Failed Blocked Not run Pass rate
1 16 14 2 0 74 88%
2 34 30 4 8 48 88%
3 51 45 6 8 31 88%
4 63 58 5 8 19 92%
5 74 70 4 6 10 95%
RUN SUMMARY · end of day 5
Area (risk) Plan Passed Failed Blocked Not run
Prescription upload (High) 20 17 3 0 0
Checkout & payment (High) 24 17 1 6 0
Order tracking (Medium) 26 22 0 0 4
Profile & notifications (Low) 20 14 0 0 6
Total 90 70 4 6 10
Open defects behind the 4 failed tests
PH-231 Critical order accepted with a rejected prescription (2 tests)
PH-236 Major PDF prescriptions cannot be uploaded (1 test)
PH-240 Minor delivery fee shown as 7.5 instead of 7.50 (1 test)
Not tested: wallet payments, 6 tests blocked (payment sandbox down since day 2)
10 Medium and Low tests not run (out of time)
Remaining risk: prescription check can be bypassed; wallet payments untestedCommon mistakes
- Counting attempts instead of tests: a test retested after a fix is counted twice, or its old failure still counts against the pass rate.
- Counting blocked tests as executed or failed, so the four counts no longer add up to the plan.
- Quietly removing skipped tests from the plan, or changing a definition mid-run, so that progress looks better than it is.
The QA perspective
Each day take the numbers from the test management tool at the same time, check that they add up to the plan, and read the new failures and blockers before writing: the risk lines come from there, not from the totals. For what a good report contains, how to tailor it to each reader and why a blocker is raised the day it appears instead of waiting for the daily status, see the lesson on test reporting.
Practice exercise
Turn a tool export into correct counts
You are testing a school fees portal. Below is the raw data at 16:00 on day 2 of a four-day regression run, and the tool export counts execution attempts, not tests. Work out the correct counts, check them against the plan, and write the counts line of the daily status.
Then say what the team would have read if you had copied the export as it is.
Run: School fees portal 2.3 · regression run · day 2 of 4 · data at 16:00
Planned: 64 tests · plan for the end of day 2: 32 executed
Tool export (one row per execution attempt, not per test)
attempts: 41 passed: 29 failed: 7 blocked: 5
From the tester's log
TC-17, TC-18, TC-19 failed on rc1 (defect FEE-512); FEE-512 was fixed and
all three passed on rc2. Both attempts of each test are in the export.
Blocked: 5 installment-plan tests; the bank test API is down (HTTP 503)
since 10:00; owner: integration team; no estimate
Open: FEE-520 Critical paying one installment marks the whole fee as paid
(2 failed tests)
FEE-523 Minor receipt date in the wrong format (2 failed tests)- Turn attempts into tests: each test counts once, with its latest result.
- Calculate executed, not run, progress and the pass rate, and check that the counts add up to 64.
- Compare your counts with the export and explain each difference.
- Write the counts line: executed against the plan, then passed, failed, blocked and not run.
Show the model answer
Counting: the 41 attempts include TC-17, TC-18 and TC-19 twice, once failed on rc1 and once passed on rc2. Keeping the latest result, 38 tests have a result: 29 passed, 4 failed (FEE-520 and FEE-523, two tests each) and 5 blocked. Not run = 64 − 38 = 26. Check: 29 + 4 + 5 + 26 = 64.
Numbers: executed = 29 + 4 = 33 of 64 (52%) against a plan of 32, so the run is on plan. Pass rate = 29 ÷ 33 = 88%. Read as it is, the export would give 41 "executed" tests and 7 failures, because it counts three tests twice and the 5 blocked attempts as if they had run. Even its passed and failed attempts alone give 29 ÷ 36 = 81%, because the three rc1 failures that were later fixed still count.
Counts line: "School fees 2.3 regression, day 2 of 4 (16:00, rc2): 33 of 64 executed (52%, plan 32); 29 passed, 4 failed, 5 blocked, 26 not run; pass rate 88% of the executed tests." The lines after it, on the Critical FEE-520, the blocked installment tests with their owner and what you need, are written as in the lesson on test reporting.
Frequently asked questions
Do blocked tests count as executed?
No. They count as not executed, together with the tests not run yet, and the status names them separately with the blocker, its owner and since when. Counting them as executed overstates progress; counting them as failed sends developers looking for defects that may not exist. Once the blocker is removed and a blocked test runs, its latest result moves it to passed or failed.
What is a good pass rate?
There is no universal target. A pass rate describes only the tests that ran, and its meaning depends on which tests they were. The exit criteria in the test plan say what must be true at the end, for example all high-risk tests passed and no open Critical defect. Track the pass rate per area and over time, and always show it next to the number of tests not executed.
Do we need a written daily status if we have a daily stand-up?
Usually yes, but keep it short. The stand-up is spoken and reaches only the people present; the written status reaches stakeholders who were not there, keeps the numbers traceable and builds the history the run summary is written from. Raise blockers at the stand-up or immediately, then confirm them in writing.
References
- ISTQB Certified Tester Foundation Level (CTFL) v4.0The foundation syllabus of software testing: test levels and types, test design techniques, test management and tools.
- ISO/IEC/IEEE 29119-3:2021 — Software and systems engineering — Software testing — Part 3: Test documentationTemplates and content for test documentation — test policy, strategy, plan, status and completion reports, incident reports.
- ISTQB GlossaryThe standard definitions of testing terms — test case, regression testing, severity, priority, boundary value analysis and the rest.
Practise with Mutqan tools
Calculate percentages, percentage change, increase/decrease and what percent X is of Y.
Normalise Markdown formatting (headings, lists, tables, emphasis) and preview the rendered result.
Convert Markdown (.md) files to PDF in your browser: headings, lists, tables, code blocks, quotes, links and images typeset with embedded fonts.
Go deeper
Your progress on this topic
Not started
Mark the topic complete when you have studied it and done the exercise.
Sign in to track your progress
Mark topics complete, continue where you left off and unlock specialisation exams.