Record all test results using Good Documentation Practices (GDP)! A summary of what to capture is provided in Appendix D.
Minimize annotations, but don't hesitate to make them if it helps clarify results or test efforts. For example, if a test is stopped at the end of one day and resumed the next day, an annotation should be made to show where and when (the day) testing stopped and where and when testing resumed.
Similarly, if a test is performed by multiple testers, a sign-off should be available for all testers. If this is not possible, then an annotation should be made indicating which steps were executed by whom, and which steps were performed by which tester.
If a screen shot is captured to provide evidence of compliance, the screen shot becomes a part of the test record. It's important, therefore, to properly annotate the screen shot. Annotations must include:
· A unique “attachment” reference (for example “Attachment 1 of VPR-ENG-001”);
· The tester's initials (or signature, if required by company procedures);
· The date the screenshot was taken;
· A reference to the test protocol and test step;
· Correct pagination in the form “page x of y” (even if a single page).
Annotations must ensure complete linkage to the originating test and are in line with GDP.
Protocol variances are to be handled as defined in the VMP. Appendix B provides a summary of standard techniques. Company policies and procedures must be followed, as always.
Issues and observations should be noted by the tester. Support tools (for example, problem reporting systems) should be provided to facilitate such reporting, but do not need to be detailed in the test report since they don't represent a test failure.
If you want to learn more about Good Documentation Practices why not
Reporting
At its core, the test report summarizes the results (from IQ, OQ, and PQ) of the test efforts. Testers and reviewers involved in testing, along with test dates, should be recorded. In general, a report follows the following outline:
I. System Description
II. Testing Summary (test dates, protocols used, testers involved) III. Results
IV. Deviations, Variances, and Incidents V. Observations and Recommendations VI. Conclusions
In many cases, there will be variances (for example, the test protocol steps or the expected results were incorrect) and/or deviations (expected results not met), which should be handled in accordance with the VMP.
Generally, variances and deviations are summarized in the test report, showing that they are recognized and being dealt with properly.
Failures, on the other hand, must be explicitly described and explained.
For each failure, provide a link to the formal problem report. It's typical to summarize the results in the “Results” section of the report and then use an appendix to provide additional details (for example, a problem report tracking number).
It's also a good practice to allow the test team to provide
situation the system runs extremely slow. Perhaps the requirements were met, but the customer or end user may not be happy with the application.
Allowing recommendations in the report can highlight areas that may need further investigation before launch. The report should draw a conclusion based on objective evidence that the product:
· Sufficiently meets the requirements
· Is safe (per verified risk mitigations)
· Can consistently fulfill its intended purpose (or not).
Observations and recommendations should be followed up by the development team.
If there are test failures, a risk assessment should be performed to determine whether the system can or cannot be used in a production environment. For example, if a requirement specifies that a response to a particular input is made within five seconds, but one response time comes back after five seconds, an assessment can be performed. If the team agrees that this is acceptable for production (and this is the only failure), the rationale can be documented in the test report (or release report), and the system can be accepted for production use. Of course, if the failure is in a safety-critical area, there's probably no reasonable rationale for releasing the system. (presuming it's justifiable).
If the validation effort is contracted out, however, the test report may not be available or appropriate to determine whether the system can be released with test failures—especially if the impact of some of the failures cannot be fully be assessed by the test team. In such cases, it's acceptable for a report addendum or a separate “release report” to be produced with the company's rationale for releasing the system (presuming it's justifiable).
Depending on the size of the system, multiple reports can be used as required. For example, a unit testing report may be issued after a batch of unit tests have completed; or on the other side of the scale there may be a requirement for each test to be reported on in its own right.
Managing the results
User requirements;
Supporting documentation (internal and external – user manuals, maintenance manuals, admin, and so on);
Vendor data (functional specification, FAT, SAT, validation protocols);
Design documentation;
Executed protocols;
· An archive copy of the software (including libraries, data, and so on);
· Protocol execution results;
· The test report;
· A known bug list with impact assessment.
Then, when the auditor asks whether the software has been validated, present the Validation Pack (or at least take them to the relevant library).
You'll make his or her day.
Testing is done, the software is released, and you're making money.
What could go wrong? Audits and auditors—the reality that everyone in a regulated industry faces. You can mitigate potential problems by preparing a Validation Pack that contains: