Test Execution and Reporting Results
Prerequisites: You should already understand Writing Effective Bug Reports. Leads to: After this, you'll be ready for Reviewing Test Cases.
Every technique and artifact this path has taught eventually converges on one moment: actually running the planned tests and telling the rest of the team what happened. Foundations' Test Strategy vs. Test Plan named exit criteria as part of a plan; this module is where those criteria actually get evaluated against real execution results, and where a release-readiness conversation gets the evidence it needs.
Why This Matters
A team that reports execution vaguely. AtlasBank's QA team finishes a testing cycle for a Loan Portal release and reports, in the release-readiness meeting, that testing "went well, mostly passed, a few issues but nothing major." Nobody in the room — not the QA lead's own manager, not the product owner deciding whether to ship — can actually evaluate this claim. How many test cases ran? How many failed, and were any of them tied to the plan's stated exit criteria? Is "nothing major" a severity judgment anyone else can verify, or just a feeling? The release ships on the strength of a vague, unverifiable summary, and when a real defect surfaces post-release in an area that had, in fact, failed during testing, nobody can quickly determine whether that failure was properly evaluated or simply lost in translation between "mostly passed" and a ship decision.
A team that reports execution precisely. A different team tracks execution status explicitly throughout the cycle — every test case marked Pass, Fail, or Blocked, with failures linked directly to logged defects. At the release-readiness meeting, the QA lead presents a specific, checkable statement: 342 of 350 planned test cases passed; 6 failed, of which 2 are Critical severity and unresolved; 2 are blocked pending a test environment fix. Exit criteria (from the test plan) required zero open Critical defects — so the answer to "are we ready" is a direct, evidence-backed "not yet," not a judgment call anyone has to trust blindly.
Both teams ran real testing. Only one of them could actually answer "are we ready to ship" with evidence instead of an impression.
What Execution Tracking and Reporting Involves
Execution status is the real-time record of each test case's outcome during a test cycle: Pass (behaved as expected), Fail (didn't match the expected result — this should link directly to a bug report, per the previous module), or Blocked (couldn't be executed at all, usually due to an environment issue, a missing dependency, or an earlier blocking defect). Tracking these three states explicitly, per test case, is what makes every other reporting artifact in this module possible.
Daily reporting is a short, regular status update during an active test cycle — not a full report, just enough for the team to track progress and react quickly to blockers. A useful daily report is brief and specific.
Worked example — a daily execution report:
| Field | Content |
|---|---|
| Date | Day 3 of 5 |
| Executed Today | 45 test cases |
| Cumulative | 210 of 350 planned test cases executed (60%) |
| Pass / Fail / Blocked (Today) | 40 / 3 / 2 |
| New Defects Logged | 3 (1 Critical, 2 Medium) |
| Blockers | Test environment for the payment-gateway integration is down; 2 test cases blocked until resolved |
| On Track for Exit Criteria? | At risk — Critical defect from Day 2 still unresolved |
Test Summary Reports are the comprehensive, end-of-cycle artifact: total test cases planned versus executed, final Pass/Fail/Blocked counts, defects found by severity, and an explicit statement of whether exit criteria (from the test plan) were met.
Metrics worth including in a summary report connect directly to Foundations' QA Metrics & Measurement: defect density for the release, pass rate trends across the cycle, and time-to-resolution for defects found — always with the same discipline that module taught, reporting these as evidence to investigate, not proof on their own.
Stakeholder communication means adapting the same underlying facts to the audience — a product owner needs a ship/no-ship-relevant summary; an engineering lead may want the defect list broken down by component; a QA lead's own report to their manager might emphasize process and capacity. The facts stay the same; what's emphasized and how much detail is included should match what the specific reader actually needs to act on.
Exit criteria evaluation is where a test summary report earns its purpose: stating explicitly whether the objective, specific criteria from the test plan (zero open Critical defects, 95% test case execution, or whatever the plan actually specified) were met — a direct, checkable answer, not a subjective impression.
A beginner reports that testing "went well" or "found some issues." A senior tester reports specific, checkable numbers tied directly back to the plan's stated exit criteria — because the entire value of an exit criterion, as Foundations' Test Strategy vs. Test Plan taught, is that it prevents exactly this kind of unfalsifiable, feel-based release conversation.
Workplace Deliverables
This module teaches the skills behind two real, recurring workplace documents:
✓ Daily Execution Report — a brief, regular status update during an active cycle (worked example above)
✓ Test Summary Report — the comprehensive end-of-cycle report evaluating exit criteria (worked example below)
(Templates for both are planned — tracked in NOT_NOW.md.)
Reviewer Checklist
Before sending a test summary report, verify:
☐ Total planned versus executed test cases stated explicitly ☐ Pass/Fail/Blocked counts are specific numbers, not general impressions ☐ Every failure links to a logged defect with severity and priority ☐ Exit criteria from the test plan are evaluated explicitly — met, or not, with specifics ☐ The report's level of detail matches its actual audience
When NOT to Use Full Formal Reporting
- Very short, informal internal tools with a single stakeholder already watching progress in real time (a shared dashboard, a live chat channel) may not need a separate formal report duplicating information already visible — but this doesn't apply once a release decision genuinely depends on a documented, checkable summary
- A tiny, single-day test cycle with only a handful of test cases may reasonably skip the full daily-report cadence — a brief end-of-day summary is proportionate
- Personal, exploratory testing sessions not tied to a release decision are already covered by session reports (from Section 4) — a separate formal execution report on top of that would be redundant
How This Works on a Real Project
AtlasBank's five-day test cycle for a Mobile App release runs with daily reports tracking progress against the plan's stated exit criteria: zero open Critical defects, 98% test case execution, full regression suite passing. By Day 4, execution is at 95%, with one Critical defect (a payment-confirmation display bug) still open. The Day 4 daily report flags this explicitly as "at risk for exit criteria" rather than waiting until Day 5's final summary to surface it — giving the team a full day to prioritize a fix rather than discovering the gap only at the point of a release decision.
Worked example — a Test Summary Report, end of cycle:
| Field | Content |
|---|---|
| Cycle | Mobile App v4.3.0, 5-day cycle |
| Test Cases Planned | 350 |
| Test Cases Executed | 350 (100%) |
| Pass / Fail / Blocked | 344 / 4 / 2 |
| Defects Found (by Severity) | 1 Critical, 3 High, 8 Medium, 12 Low |
| Exit Criteria Met? | No — 1 Critical defect (payment-confirmation display) remains open; all other criteria met |
| Recommendation | Do not release until the Critical defect is resolved and retested; all other areas are release-ready |
This report gives the release-readiness meeting exactly what the opening scenario's vague version couldn't: a specific, checkable answer, with a clear recommendation grounded in the plan's own stated criteria — not a feeling anyone has to simply trust.
Common Mistakes
Mistake 1: Reporting execution progress or results in vague, unfalsifiable terms. "Mostly passed" and "nothing major" can't be verified or acted on — specific counts tied to exit criteria can.
Mistake 2: Waiting until the final summary to surface a risk to exit criteria. The Mobile App example's Day 4 flag is what gives the team time to react — surfacing the same risk only in the Day 5 summary removes any chance to respond before the release decision.
Mistake 3: Treating every report the same regardless of audience. A product owner and an engineering lead need the same underlying facts presented differently — a one-size-fits-all report often under-serves both.
Mistake 4: Failing to state exit criteria evaluation explicitly. A summary report that lists numbers without explicitly connecting them back to "were exit criteria met" leaves the actual release-readiness question unanswered.
Best Practices
Practice 1: Track Pass/Fail/Blocked explicitly per test case throughout the cycle, not just at the end. This is what makes every other reporting artifact in this module possible — real-time tracking, not end-of-cycle reconstruction.
Practice 2: Surface exit-criteria risk as soon as it's visible, not just in the final report. Early visibility is what gives a team time to actually respond, as the Mobile App example shows.
Practice 3: State exit criteria evaluation explicitly and directly in every summary report. This is the single most important sentence in the report — everything else supports this specific, checkable answer.
Practice 4: Adapt detail and emphasis to the actual audience, without changing the underlying facts. The same real numbers, presented with the right emphasis for who's reading, serve every stakeholder better than one generic report.
On a fintech platform, a QA team's end-of-cycle reports consistently described testing as "successful" even when several Medium-severity defects remained open, because the team's informal convention was that only Critical defects counted as blocking. A new engineering lead, unfamiliar with this unstated convention, approved a release believing "successful" meant no open defects at all, and was blindsided when several Medium-severity issues surfaced in production shortly after. The team's reports weren't dishonest — but leaving the actual exit criteria implicit, rather than stating them explicitly in every report, let two different readers draw two very different conclusions from the same word.
Mini Challenge
Scenario: A three-day test cycle for AtlasBank's Admin Portal has just finished. 120 of 125 planned test cases executed (5 were blocked by an environment issue that was never resolved); 115 passed, 5 failed (2 Medium, 3 Low severity, all logged as defects). The test plan's exit criteria required 100% execution and zero open Medium-or-higher defects.
Your task: Write a complete Test Summary Report for this cycle, including an explicit, direct answer to whether exit criteria were met.
Key Takeaways
- Execution status (Pass/Fail/Blocked), tracked explicitly per test case, is what makes every other reporting artifact in this module possible.
- A test summary report's most important sentence is its explicit, direct evaluation of exit criteria — met, or not, with specifics.
- Surfacing exit-criteria risk as soon as it's visible, not just at the final report, gives a team real time to respond.
- The same underlying facts should be adapted in emphasis and detail for different stakeholders, without changing what's actually true.
What You Just Learned
- The difference between Pass, Fail, and Blocked, and why tracking them explicitly matters
- How to write a daily execution report and a comprehensive test summary report
- Why an explicit, direct exit-criteria evaluation is the single most important sentence in any summary report
- How a fintech team's unstated, implicit exit-criteria convention led two readers to two different conclusions from the same word
Next: Reviewing Test Cases
Related Topics
- Test Strategy vs. Test Plan — Where exit criteria are originally defined, evaluated directly by this module's reports
- QA Metrics & Measurement — The metrics discipline this module's reporting connects to directly
- Writing Effective Bug Reports — Where individual failures get documented, feeding directly into this module's Fail counts
Interview Questions
Q1: What would you include in an end-of-cycle test summary report?
What to look for: A candidate who names specific fields (execution counts, Pass/Fail/Blocked breakdown, defects by severity, and critically, an explicit exit-criteria evaluation) — not a vague "a summary of how testing went."
Q2: How do you communicate testing status differently to a product owner versus an engineering lead?
What to look for: A candidate who describes adapting emphasis and detail to the audience's actual decision-making needs, while keeping the underlying facts identical — not two genuinely different, potentially conflicting stories.
Many candidates describe a good test report as one that's "comprehensive" or "detailed," without naming what specifically makes it actionable. That's incomplete — the highest-value content in any test report is the explicit, direct exit-criteria evaluation, not just volume of information. A strong answer names this specifically as the report's most important element.
Q3: A test cycle is at risk of missing its exit criteria three days before the planned release. What do you do?
What to look for: A candidate who describes surfacing this immediately and explicitly, not waiting for the final report — connecting directly to this module's emphasis on early visibility over end-of-cycle surprises.
Glossary
Execution Status: The tracked outcome of a test case during a test cycle — Pass, Fail, or Blocked.
Blocked: A test case that couldn't be executed at all, typically due to an environment issue, missing dependency, or an earlier blocking defect.
Test Summary Report: The comprehensive, end-of-cycle report stating execution totals, defect counts by severity, and an explicit exit-criteria evaluation.
Exit Criteria Evaluation: A direct, specific statement of whether a test plan's stated exit criteria were met, the core purpose of a test summary report.
Quick Revision
Remember these five points:
✓ Track Pass/Fail/Blocked explicitly per test case — this is the foundation every other reporting artifact depends on.
✓ A test summary report's most important sentence is its explicit exit-criteria evaluation.
✓ Surface exit-criteria risk as soon as it's visible, not just in the final report.
✓ Adapt detail and emphasis to the audience, without changing the underlying facts.
✓ Vague terms like "mostly passed" or "nothing major" can't be verified or acted on — specific, checkable numbers can.