Test Summary Report
What is a Test Summary Report?
A Test Summary Report (TSR) is a formal document produced at the end of the testing phase that consolidates all testing activities, results, metrics, defect statistics, and sign-offs for a release. It is the final quality gate document that must be approved before a release is promoted to production.
The TSR answers the question: "Is this release ready for production?"
When to Produce a TSR
A TSR is produced for every planned release:
| Release Type | TSR Required? |
|---|---|
| Major / Minor release | ✅ Full TSR required |
| Patch / Bugfix release | ✅ Abbreviated TSR required |
| Hotfix (P1 resolution) | ⚠️ Lightweight TSR within 24h of deployment |
| Internal tooling / non-customer facing | Optional |
TSR Template
1. Release Information
| Field | Value |
|---|---|
| Project Name | Transaction Service |
| Release Version | v1.2.0 |
| Test Environment | UAT (uat.yourapp.com) |
| Testing Period | 2024-03-01 – 2024-03-14 |
| Report Date | 2024-03-15 |
| Prepared By | QA Lead — Jane Smith |
| Approved By | Product Manager — John Doe |
2. Executive Summary
Provide a concise paragraph summarising overall test results, confidence level, and the recommendation (approve for release / conditional approval / hold).
Example:
Testing for release v1.2.0 of Transaction Service has been completed across unit, integration, regression, end-to-end, and performance test suites. All 847 test cases were executed. 843 passed; 4 were deferred as known P3 issues with accepted workarounds. No P1 or P2 defects remain open. Performance benchmarks are met with p99 latency at 312ms (target: < 500ms). The QA team recommends this release for production deployment.
3. Scope of Testing
In Scope
- Transaction listing with date range filtering (US-105 to US-112)
- CSV export of transaction history (US-113 to US-115)
- Pagination and sorting controls (US-116 to US-118)
- Authentication and authorisation (US-101 to US-104)
- Regression of all existing transaction features
Out of Scope
- Payment gateway integration (covered by separate Payment Service TSR)
- Mobile application UI (covered by Mobile team's TSR)
- Batch processing jobs (scheduled for v1.3.0)
4. Test Execution Summary
4.1 Test Case Results
| Test Type | Total | Passed | Failed | Blocked | Skipped | Pass Rate |
|---|---|---|---|---|---|---|
| Unit Tests | 412 | 412 | 0 | 0 | 0 | 100% |
| Integration Tests | 186 | 185 | 0 | 0 | 1 | 99.5% |
| Regression Tests | 158 | 155 | 2 | 1 | 0 | 98.1% |
| End-to-End Tests | 64 | 63 | 0 | 1 | 0 | 98.4% |
| Performance Tests | 27 | 28 | 0 | 0 | 0 | 100% |
| Total | 847 | 843 | 2 | 2 | 1 | 99.5% |
4.2 Code Coverage
| Module | Line Coverage | Branch Coverage | Status |
|---|---|---|---|
api (controllers) | 91% | 87% | ✅ |
domain (services) | 96% | 93% | ✅ |
infrastructure (repos) | 84% | 79% | ✅ |
common (utils/exceptions) | 88% | 82% | ✅ |
| Overall | 91% | 86% | ✅ Above 80% threshold |
4.3 Test Execution Timeline
| Phase | Start | End | Duration | Environment |
|---|---|---|---|---|
| Unit + Integration CI | 2024-03-01 | 2024-03-08 | Continuous | DEV |
| Regression testing | 2024-03-09 | 2024-03-11 | 3 days | SIT |
| UAT + E2E testing | 2024-03-12 | 2024-03-13 | 2 days | UAT |
| Performance testing | 2024-03-14 | 2024-03-14 | 1 day | PERF |
5. Defect Summary
5.1 Defect Statistics
| Severity | Raised | Fixed | Deferred | Open |
|---|---|---|---|---|
| P1 — Critical | 0 | 0 | 0 | 0 |
| P2 — High | 2 | 2 | 0 | 0 |
| P3 — Medium | 7 | 3 | 4 | 0 |
| P4 — Low | 3 | 0 | 3 | 0 |
| Total | 12 | 5 | 7 | 0 |
5.2 Open / Deferred Defects
| Defect ID | Severity | Summary | Reason for Deferral | Target Fix Version |
|---|---|---|---|---|
| JIRA-441 | P3 | CSV export trailing comma on last column | Low impact — workaround in Excel | v1.2.1 |
| JIRA-443 | P3 | Date picker shows previous month on first load | UX only, no data impact | v1.2.1 |
| JIRA-447 | P3 | Empty state message not localised for FR locale | FR locale not yet active in prod | v1.3.0 |
| JIRA-452 | P3 | Pagination total count off by 1 on empty filter | Edge case, no data loss | v1.2.1 |
| JIRA-455 | P4 | Console warning in Firefox 121 dev tools | Dev tools only | v1.3.0 |
| JIRA-457 | P4 | Tooltip alignment off by 2px in Safari | Cosmetic | v1.3.0 |
| JIRA-461 | P4 | Sort icon missing on mobile viewport | Mobile styles — tracked separately | v1.3.0 |
Decision: All deferred defects are P3/P4 with no data integrity impact. Product Manager has accepted deferral. Release is approved to proceed.
5.3 Defect Density
| Module | LOC Changed | Defects Found | Defect Density |
|---|---|---|---|
| Transaction API | 1,240 | 5 | 4.0 per KLOC |
| Export Feature | 380 | 4 | 10.5 per KLOC |
| Auth Module | 210 | 0 | 0 per KLOC |
| Common Utils | 95 | 3 | 31.6 per KLOC ⚠️ |
Note: High defect density in
common/utils— Tech Lead to schedule refactoring in v1.3.0.
6. Performance Test Results
| Metric | Target | Actual | Status |
|---|---|---|---|
| p50 latency | < 50ms | 28ms | ✅ |
| p95 latency | < 200ms | 156ms | ✅ |
| p99 latency | < 500ms | 312ms | ✅ |
| Throughput | ≥ 1000 RPS | 1,247 RPS | ✅ |
| Error rate | < 0.1% | 0.02% | ✅ |
| JVM heap (steady) | < 512MB | 387MB | ✅ |
| CPU (steady) | < 60% | 44% | ✅ |
| Memory leak (30min soak) | None | None detected | ✅ |
Performance verdict: All NFR targets met. ✅
7. Security Testing Results
| Check | Tool | Result |
|---|---|---|
| OWASP Dependency Check | mvn dependency-check:check | ✅ No critical CVEs |
| Static analysis (Sonar) | SonarQube | ✅ 0 blocker, 0 critical issues |
| Dynamic scan (DAST) | OWASP ZAP | ✅ No high/critical findings |
| SQL injection | Automated + manual | ✅ No vulnerabilities |
| Auth bypass attempts | Manual pen-test | ✅ All attempts blocked |
8. Environment and Configuration
| Environment | Version Tested | Database Version | Notes |
|---|---|---|---|
| SIT | v1.2.0-rc1 | PostgreSQL 16.1 | Seeded with synthetic data |
| UAT | v1.2.0-rc2 | PostgreSQL 16.1 | Anonymised prod data snapshot |
| PERF | v1.2.0-rc2 | PostgreSQL 16.1 | 3-month production volume data |
Flyway migrations applied:
V1_2_0__add_transaction_category_column.sql✅ Applied and verified
9. Test Artifacts
| Artifact | Location |
|---|---|
| JaCoCo coverage report | target/site/jacoco/index.html |
| Surefire unit test report | target/surefire-reports/ |
| Failsafe integration test report | target/failsafe-reports/ |
| Gatling performance report | target/gatling/transactionservice/index.html |
| SonarQube dashboard | SonarQube Project Link |
| Defect log (Jira) | Jira Query — v1.2.0 bugs |
10. Risks and Mitigations
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| P3 export bug (JIRA-441) causes user confusion | Low | Low | Release note included; fix in v1.2.1 |
| High load on first day post-release | Medium | Medium | Auto-scaling configured; perf tests passed |
| Flyway migration slow on large prod tables | Low | High | Tested against prod-volume data; 1.2s execution |
11. Sign-off
| Role | Name | Decision | Date |
|---|---|---|---|
| QA Lead | Jane Smith | ✅ Approved | 2024-03-15 |
| Tech Lead | Bob Johnson | ✅ Approved | 2024-03-15 |
| Product Manager | John Doe | ✅ Approved — proceed to deploy | 2024-03-15 |
| Security Engineer | Alice Chen | ✅ Approved | 2024-03-14 |
12. Recommendation
✅ APPROVED FOR PRODUCTION DEPLOYMENT
Release v1.2.0 of Transaction Service meets all quality gates. No P1 or P2 defects are open. All performance NFRs are satisfied. Deferred P3/P4 defects have been reviewed and accepted by the Product Manager. The team is ready to proceed with deployment per the deployment runbook.
Planned deployment window: 2024-03-15 22:00 UTC On-call engineer: @oncall-bob Rollback owner: @oncall-alice
TSR Quality Gates
Before the TSR can be signed off, these gates must all be green:
- Unit test pass rate = 100%
- Integration test pass rate ≥ 99%
- Code coverage ≥ 80% overall
- Sonar quality gate green (0 blockers, 0 criticals)
- 0 open P1 defects
- 0 open P2 defects
- All P3 defects reviewed and accepted/deferred by PM
- Performance NFRs met (p99, throughput, error rate)
- Security scan completed with no high/critical findings
- All sign-offs obtained
Use your CI/CD pipeline to auto-generate the test statistics section from JaCoCo XML reports, Surefire XML output, and SonarQube API. Only the narrative summary, defect table, and sign-offs need to be filled in manually. This saves 1–2 hours per release.