[ISTQB] 5.3 Test Progress Monitoring and Control (K2)

Test monitoring is essential for giving relevant feedback and visibility about test activities (progress).

LO-5.3.1 Recall common metrics used for monitoring test preparation and execution (K1)

• Information gathered for monitoring and providing feedback can be collected manually or automatically. 
• Information can be captured from exit criteria (coverage, number of faults open, severity) and metrics may also be used against planned schedule and budget. 

During test preparation, managers would likely to know how much test cases have been prepared against how much test cases have been planned to be elaborated.

During test execution, it's the number of test executed which is monitored (run/not run, passed/failed) against test cases execution that have been initially planned.

LO-5.3.2 Explain and compare test metrics for test reporting and test control (e.g., defects found and fixed, and tests passed and failed) related to purpose and use (K2)

Generally, managers want to be aware on defect information (defect density, found, fixed, retest results) to assess improvements on quality, risks, re-evaluate test costs and test milestones.

Metrics should be collected during and at the end of a test level in order to assess adequacy of test objectives and approach plus its effectiveness.

Then these needs to be analysed against planned goals for elaborating test controls actions that would correct any deviation and take decisions (re prioritize tests, change test schedule, set a new entry criteria).

LO-5.3.3 Summarize the purpose and content of the test summary report document according to the 'Standard for Software Test Documentation' (IEEE Std 829-1998) (K2)

Monitoring test progress thorough testing activities, facilitates the elaboration of the test summary report.

According to the IEEE Std 829-1998, a test summary:
- reports lists and specifies the identifier (responsible of the report).
- includes the summary info such as dates when exit criteria were met, defect analysis, test logs...
- lists variances from the designed test plan.
- shows comprehensive assessment (combinations not tested and why), summary results, evaluation (item limitation, risks of failure),
- summaries activities, machines, resources, people time used and any approvals made.