Test Summary Report
What is a Test Summary Report?โ
A Test Summary Report (TSR) is a formal document produced at the end of the testing phase that consolidates all testing activities, results, metrics, defect statistics, and sign-offs for a release. It is the final quality gate document that must be approved before a release is promoted to production.
The TSR answers the question: "Is this release ready for production?"
When to Produce a TSRโ
A TSR is produced for every planned release:
| Release Type | TSR Required? |
|---|---|
| Major / Minor release | โ Full TSR required |
| Patch / Bugfix release | โ Abbreviated TSR required |
| Hotfix (P1 resolution) | โ ๏ธ Lightweight TSR within 24h of deployment |
| Internal tooling / non-customer facing | Optional |
TSR Templateโ
1. Release Informationโ
| Field | Value |
|---|---|
| Project Name | Transaction Service |
| Release Version | v1.2.0 |
| Test Environment | UAT (uat.yourapp.com) |
| Testing Period | 2024-03-01 โ 2024-03-14 |
| Report Date | 2024-03-15 |
| Prepared By | QA Lead โ Jane Smith |
| Approved By | Product Manager โ John Doe |
2. Executive Summaryโ
Provide a concise paragraph summarising overall test results, confidence level, and the recommendation (approve for release / conditional approval / hold).
Example:
Testing for release v1.2.0 of Transaction Service has been completed across unit, integration, regression, end-to-end, and performance test suites. All 847 test cases were executed. 843 passed; 4 were deferred as known P3 issues with accepted workarounds. No P1 or P2 defects remain open. Performance benchmarks are met with p99 latency at 312ms (target: < 500ms). The QA team recommends this release for production deployment.
3. Scope of Testingโ
In Scopeโ
- Transaction listing with date range filtering (US-105 to US-112)
- CSV export of transaction history (US-113 to US-115)
- Pagination and sorting controls (US-116 to US-118)
- Authentication and authorisation (US-101 to US-104)
- Regression of all existing transaction features
Out of Scopeโ
- Payment gateway integration (covered by separate Payment Service TSR)
- Mobile application UI (covered by Mobile team's TSR)
- Batch processing jobs (scheduled for v1.3.0)
4. Test Execution Summaryโ
4.1 Test Case Resultsโ
| Test Type | Total | Passed | Failed | Blocked | Skipped | Pass Rate |
|---|---|---|---|---|---|---|
| Unit Tests | 412 | 412 | 0 | 0 | 0 | 100% |
| Integration Tests | 186 | 185 | 0 | 0 | 1 | 99.5% |
| Regression Tests | 158 | 155 | 2 | 1 | 0 | 98.1% |
| End-to-End Tests | 64 | 63 | 0 | 1 | 0 | 98.4% |
| Performance Tests | 27 | 28 | 0 | 0 | 0 | 100% |
| Total | 847 | 843 | 2 | 2 | 1 | 99.5% |
4.2 Code Coverageโ
| Module | Line Coverage | Branch Coverage | Status |
|---|---|---|---|
api (controllers) | 91% | 87% | โ |
domain (services) | 96% | 93% | โ |
infrastructure (repos) | 84% | 79% | โ |
common (utils/exceptions) | 88% | 82% | โ |
| Overall | 91% | 86% | โ Above 80% threshold |
4.3 Test Execution Timelineโ
| Phase | Start | End | Duration | Environment |
|---|---|---|---|---|
| Unit + Integration CI | 2024-03-01 | 2024-03-08 | Continuous | DEV |
| Regression testing | 2024-03-09 | 2024-03-11 | 3 days | SIT |
| UAT + E2E testing | 2024-03-12 | 2024-03-13 | 2 days | UAT |
| Performance testing | 2024-03-14 | 2024-03-14 | 1 day | PERF |
5. Defect Summaryโ
5.1 Defect Statisticsโ
| Severity | Raised | Fixed | Deferred | Open |
|---|---|---|---|---|
| P1 โ Critical | 0 | 0 | 0 | 0 |
| P2 โ High | 2 | 2 | 0 | 0 |
| P3 โ Medium | 7 | 3 | 4 | 0 |
| P4 โ Low | 3 | 0 | 3 | 0 |
| Total | 12 | 5 | 7 | 0 |
5.2 Open / Deferred Defectsโ
| Defect ID | Severity | Summary | Reason for Deferral | Target Fix Version |
|---|---|---|---|---|
| JIRA-441 | P3 | CSV export trailing comma on last column | Low impact โ workaround in Excel | v1.2.1 |
| JIRA-443 | P3 | Date picker shows previous month on first load | UX only, no data impact | v1.2.1 |
| JIRA-447 | P3 | Empty state message not localised for FR locale | FR locale not yet active in prod | v1.3.0 |
| JIRA-452 | P3 | Pagination total count off by 1 on empty filter | Edge case, no data loss | v1.2.1 |
| JIRA-455 | P4 | Console warning in Firefox 121 dev tools | Dev tools only | v1.3.0 |
| JIRA-457 | P4 | Tooltip alignment off by 2px in Safari | Cosmetic | v1.3.0 |
| JIRA-461 | P4 | Sort icon missing on mobile viewport | Mobile styles โ tracked separately | v1.3.0 |
Decision: All deferred defects are P3/P4 with no data integrity impact. Product Manager has accepted deferral. Release is approved to proceed.
5.3 Defect Densityโ
| Module | LOC Changed | Defects Found | Defect Density |
|---|---|---|---|
| Transaction API | 1,240 | 5 | 4.0 per KLOC |
| Export Feature | 380 | 4 | 10.5 per KLOC |
| Auth Module | 210 | 0 | 0 per KLOC |
| Common Utils | 95 | 3 | 31.6 per KLOC โ ๏ธ |
Note: High defect density in
common/utilsโ Tech Lead to schedule refactoring in v1.3.0.
6. Performance Test Resultsโ
| Metric | Target | Actual | Status |
|---|---|---|---|
| p50 latency | < 50ms | 28ms | โ |
| p95 latency | < 200ms | 156ms | โ |
| p99 latency | < 500ms | 312ms | โ |
| Throughput | โฅ 1000 RPS | 1,247 RPS | โ |
| Error rate | < 0.1% | 0.02% | โ |
| JVM heap (steady) | < 512MB | 387MB | โ |
| CPU (steady) | < 60% | 44% | โ |
| Memory leak (30min soak) | None | None detected | โ |
Performance verdict: All NFR targets met. โ
7. Security Testing Resultsโ
| Check | Tool | Result |
|---|---|---|
| OWASP Dependency Check | mvn dependency-check:check | โ No critical CVEs |
| Static analysis (Sonar) | SonarQube | โ 0 blocker, 0 critical issues |
| Dynamic scan (DAST) | OWASP ZAP | โ No high/critical findings |
| SQL injection | Automated + manual | โ No vulnerabilities |
| Auth bypass attempts | Manual pen-test | โ All attempts blocked |
8. Environment and Configurationโ
| Environment | Version Tested | Database Version | Notes |
|---|---|---|---|
| SIT | v1.2.0-rc1 | PostgreSQL 16.1 | Seeded with synthetic data |
| UAT | v1.2.0-rc2 | PostgreSQL 16.1 | Anonymised prod data snapshot |
| PERF | v1.2.0-rc2 | PostgreSQL 16.1 | 3-month production volume data |
Flyway migrations applied:
V1_2_0__add_transaction_category_column.sqlโ Applied and verified
9. Test Artifactsโ
| Artifact | Location |
|---|---|
| JaCoCo coverage report | target/site/jacoco/index.html |
| Surefire unit test report | target/surefire-reports/ |
| Failsafe integration test report | target/failsafe-reports/ |
| Gatling performance report | target/gatling/transactionservice/index.html |
| SonarQube dashboard | SonarQube Project Link |
| Defect log (Jira) | Jira Query โ v1.2.0 bugs |
10. Risks and Mitigationsโ
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| P3 export bug (JIRA-441) causes user confusion | Low | Low | Release note included; fix in v1.2.1 |
| High load on first day post-release | Medium | Medium | Auto-scaling configured; perf tests passed |
| Flyway migration slow on large prod tables | Low | High | Tested against prod-volume data; 1.2s execution |
11. Sign-offโ
| Role | Name | Decision | Date |
|---|---|---|---|
| QA Lead | Jane Smith | โ Approved | 2024-03-15 |
| Tech Lead | Bob Johnson | โ Approved | 2024-03-15 |
| Product Manager | John Doe | โ Approved โ proceed to deploy | 2024-03-15 |
| Security Engineer | Alice Chen | โ Approved | 2024-03-14 |
12. Recommendationโ
โ APPROVED FOR PRODUCTION DEPLOYMENT
Release v1.2.0 of Transaction Service meets all quality gates. No P1 or P2 defects are open. All performance NFRs are satisfied. Deferred P3/P4 defects have been reviewed and accepted by the Product Manager. The team is ready to proceed with deployment per the deployment runbook.
Planned deployment window: 2024-03-15 22:00 UTC On-call engineer: @oncall-bob Rollback owner: @oncall-alice
TSR Quality Gatesโ
Before the TSR can be signed off, these gates must all be green:
- Unit test pass rate = 100%
- Integration test pass rate โฅ 99%
- Code coverage โฅ 80% overall
- Sonar quality gate green (0 blockers, 0 criticals)
- 0 open P1 defects
- 0 open P2 defects
- All P3 defects reviewed and accepted/deferred by PM
- Performance NFRs met (p99, throughput, error rate)
- Security scan completed with no high/critical findings
- All sign-offs obtained
Use your CI/CD pipeline to auto-generate the test statistics section from JaCoCo XML reports, Surefire XML output, and SonarQube API. Only the narrative summary, defect table, and sign-offs need to be filled in manually. This saves 1โ2 hours per release.