phases and documents. Where appropriate the Test Strategy should clarify the project specific relationships. The various system Test Specifications are related to other documents in the life cycle and either use information from those documents as input (reference) data, or are in turn referred to by other documents and therefore provide output (result) data.
These related phases and documents are:
5.6.1 Validation Plan and Project Quality Plan
Where they include the Test Strategy, the Validation Plan or Project Quality Plan should explicitly indicate which specifications and corresponding Test Specifications should be produced (and tests conducted).
22 Testing Computer Systems for FDA/MHRA Compliance
As a minimum, the Validation Plan should refer to the process of auditing the supplier to ensure that supplier tests are properly conducted and may also reference a supplier audit report that indicates the general nature and scope of these tests.
However, at the time of writing the Validation Plan for a large or complex system it is unlikely that the user will have sufficient idea of the system to be used to be able to define the tests in much detail (unless the user is implementing the system themselves).
The detailed requirements of the Test Specifications will more usually be deferred to the Project Quality Plan. In the case of large or complex projects a separate Test Strategy document may be produced, or the content may be included in the IQ, OQ and PQ Protocols.
Note that these interrelationships are summarised in a traceability matrix in Appendix G – Traceability Matrices.
5.6.2 Design Specification(s)
The various Design Specification(s) provide an increasing level of detail regarding the function and design of the system.
Design Specifications should be written in a structured manner, so that each separate function of the system is clearly described and can be individually tested. They should contain explicit, concrete details of functionality and design which can be tested and pass/fail criteria should be clearly identified — rather than implied).
As an example The system will interface to a Schmidt Model 32X bar code scanner, capable of scanning and identifying 15 pallets per minute on a continuous basis rather than The system shall be capable of identifying a large number of pallets.
When writing Design Specifications it is useful if:
• The corresponding Test Specification is written in parallel, almost certainly by a different person or team.
• A member of the test team reviews the Design Specification.
Both of these steps help ensure that functional and detailed design requirements are testable.
The relationship between a Design Specification, the corresponding Test Specification, and the Test Scripts should be identified as linked configuration items in whatever Configuration Management system is used. This ensures that if one document is changed the other(s) will be identified as part of any change control impact analysis and noted as requiring review and possible modification.
Note that these interrelationships are summarised in a traceability matrix in Appendix G – Traceability Matrices.
5.6.3 Tested Software and Hardware
Once tested at a low level (hardware and software module), the system hardware and software are a direct input to subsequent System Tests.
The tested software modules are outputs from the System Module Tests and inputs to the System Integration Testing.
The hardware is an output from the hardware testing and is an input to the System Acceptance Testing, along with the tested software.
Prior to conducting the System Acceptance Tests the system software should have successfully completed a thorough and challenging Software Integration Test. Likewise, the system hardware should have successfully completed Hardware Acceptance Testing. The
The Test Strategy 23
Figure 5.3 – Output Tested Hardware and Software as Inputs to Subsequent Tests.
purpose of the System Acceptance Test is to bring the actual system software and hardware together, and prove the overall functioning of the system in line with the requirements of the Functional Specification.
Since the previous Software Module and Software Integration Testing will have been conducted on the same (or functionally equivalent) hardware, the System Acceptance Testing should be a formal opportunity to demonstrate and document the overall functionality rather than conducting rigorous challenge tests (Figure 5.3).
It must be stressed that System Acceptance Tests should only be conducted once the underlying software and hardware have been tested and approved.
Note that these interrelationships are summarised in a traceability matrix in Appendix G — Traceability Matrices.
5.6.4 System Test Specification(s)
The System Test Specifications are used as inputs (reference documents) during the actual System Acceptance Testing.
They contain important information (as described below) and the System Test Specifications are therefore a mandatory document for conducting the System Tests. No tests can proceed until the relevant Test Specification document is reviewed and approved.
Note that these interrelationships are summarised in a traceability matrix in Appendix G – Traceability Matrices.
5.6.5 Factory/Site Acceptance Test Results and IQ, OQ and PQ
There is an increasing tendency to acknowledge that IQ, IQ and PQ Protocols and Reports (which have been adopted from process/equipment qualification) may not be best structured to report on a complex set of interrelated computer systems tests.
The focus should always be on documenting the rationale for the scope, nature and level of testing and on interpreting the test results. In this context Test Policies, Test Strategies and supplier FATs, SATs and User Acceptance Testing may serve a more useful purpose than more traditional IQ, OQ and PQs.
Installation Qualification protocols and reports are still a useful way of documenting the installed system and of bringing the system under Configuration Management. OQ and PQ are less useful and may be omitted if they serve no useful purpose (and if the validation policy allows this).
24 Testing Computer Systems for FDA/MHRA Compliance
Where an organisation still requires a formal IQ, OQ and PQ to be conducted and reported upon, the emphasis should be on reducing the testing required solely to produce such documents. As described above, there may be a clear relationship between Factory and Site Acceptance Test results and the formal IQ, OQ and PQ Reports. Wherever possible, the IQ, OQ and PQ protocols should simply reference Test Strategies and Test Cases and IQ, OQ and PQ reports should reference the documented results of FATs and SATs.
Note that these interrelationships are summarised in a traceability matrix in Appendix G — Traceability Matrices.
The Test Strategy 25
CHAPTER 6
The Development Life Cycle of a Test Specification
As with any formal process, there is a logical sequence to be followed when developing the System Test Specifications and when conducting the System Tests and there are recommended activities that should be included in order to assure successful completion of the testing phase.
These are described in the various sections in this chapter:
6.1 Recommended Phasing; Interfaces between and the Dependencies of Activities