Final report

The purpose of this report is to obtain insight into the way the test process has gone and to document empirical data for the purposes of future test processes. 

The final report is created after issuing the release advice, usually when the test object has already been released to the next phase. More time is therefore available for it. 

The contents list of a final report is more or less the same as that of a progress report:

1. Evaluation of the test object (BDTM: Result)
   1. Status per characteristic/object part 
   2. Status of test goals 
2. Product risk and strategy adjustment (BDTM: Risks)
3. Release advice (BDTM: Result, Risks)
4. Evaluation of the test process
   1. Progress (BDTM: Time and Costs) 
   2. Quality of the test process (BDTM: all the aspects) 
5. Recommendations for future tests
6. Empirical data (optional)
7. Costs/benefits analysis (optional)

However, whereas a progress report looks ahead, the final report looks back. In other words, it mainly concerns the difference between the original plan and the final realisation. What degree of deviation is there from the original plan? Was the plan a good one, or were issues wrongly estimated? Were adjustments always timely and effective? To what extent were the preconditions met, and met promptly enough? Could bottlenecks have been prevented? These differences are analysed in particular for purposes of the risk analysis, test strategy, estimate and planning. The quality of the test process is also considered: were the chosen procedures, tools and techniques used correctly and was the test environment satisfactory? Recommendations are provided, if possible, for future tests.

The activity ‘Evaluating the test process” supplies the input for this evaluation. Also, use can be made of the “Test process evaluation” checklist. In addition, empirical data may be collected and made available to the client, or, even better, to a Testing line organisation. A last, optional, part of the final report is a costs/benefits analysis. 

The final report is made available to the client and other stakeholders, possibly by means of a presentation.

In more detail
Progress report versus final report

Although the terms ‘interim report’ or ‘progress report‘ may suggest that these are less important than the final report, in fact the opposite is true. The progress report supplies early information and advice, with which the recipients (such as client, project manager and others) can often make timely adjustments for keeping the total process on the right track. The final report is more a retrospective evaluation that mainly benefits subsequent test processes and projects.

In more detail

Empirical data

Examples of empirical data are: Size of the test object Development effort Number of defects Duration and hours per main activity Duration and hours required for specifying tests Duration and hours required to execute the tests Number of test cases Analysis of lead time per defect Number of defects to be expected Number of retests. A comprehensive summary of the empirical data that can be collected is included in the list “Metrics list”. This chapter also discusses the Goal-Question Metric method for implementing metrics.

Costs/benefits analysis

The costs of the test process are relatively simple to establish. Bear in mind, for example, the costs of the used resources, manpower and equipment. The benefits of the test process, however, are more difficult to establish. There are four types of benefits of testing. It is difficult, but not impossible, to provide a quantitative indication of these.