Simulation of executable Design Models

Executing a simulation of a Design Model may allow problems with the design and possibly with its higher-level requirements to be detected early in the software development process and to be corrected even before the production of the Source Code and the testing of the Executable Object Code.

Where the tools used to produce a Design Model permit that design to be executed within a simulator, simulation cases and procedures may be developed against the higher-level requirements in order to support the verification of the Design Model (cf. 23.2.7). To this purpose, the guidance in section 23.2.8.1 below should be followed.

Under certain conditions and with proper planning, it may also be possible to take some credit from simulation activities to support the verification of the Executable Object Code. To this purpose, the guidance in section 23.2.8.2 should be followed. However, as simulation involves a different environment and potentially different source code or executable object code from those used when testing the Source Code and Executable Object Code on the target processor, it is not possible to claim credit for the use of simulation instead of conducting Hardware/Software integration testing activities.

23.2.8.1

Simulation for verification of the Design Models (reviews and analyses)

Simulation of the Design Model against the higher-level requirements can provide some evidence as to whether the Design Model complies with the higher-level requirements, which can be used to show compliance with some of the objectives in ED-12B / DO-178B Table A-4 (typically objectives 1, 2, 4, 7, 8, 9, 11) or some objectives of ED-79 / ARP4754 section 7 or ED-79A / ARP4574A section 5.4. Other objectives such as compatibility with the target computer, conformance to standards or partitioning integrity cannot be demonstrated through simulation and should be fulfilled by means of conventional reviews and analyses. To this purpose, the applicant should:

a. Determine precisely in the PSAC which objectives related to reviews and analyses are planned to be covered by simulation and justify how the identified simulation activity fulfils those objectives.

b. Ensure that the simulation cases and procedures are developed and reviewed according to the guidance in section 23.2.8.3.

23.2.8.2

Simulation for verification of the Executable Object Code (testing)

It may be possible to take some credit from executing a simulation of the Design Model in order to support compliance with objectives 1 and 2 of ED-12B / DO-178B Table A-6 and objective 3 of Table A-7.

Note: Since simulation cases should be based on the Higher-level Requirements, compliance with objectives 3 and 4 of Table A-6 cannot be wholly or partly claimed based on the use of simulation of the Design Model. Similarly, compliance with objective 5 of Table A-6 cannot be wholly or partly claimed based on the use of simulation of the Design Model because this objective is related to hardware compatibility aspects.

To this purpose, the applicant should:

a. Ensure that the Design Model used for simulation is identical to the one used to produce the Source Code.

b. Determine precisely in the PSAC which testing objectives are planned to be covered by simulation and justify how the identified simulation activity fulfils those objectives. c. Perform an analysis to identify any differences between the target environment and the simulation environment and provide a rationale for why these differences are acceptable.

d. Ensure that there are no differences between the source code used for simulation and the Source Code of the final software product. If there are any differences, they should be minor and should be justified and a rationale provided for why they are acceptable.

e. Note: For test coverage objectives (ED-12B / DO-178B objectives 5, 6, 7 and 8 of Table A-7), credit can be claimed only if the source code used for simulation and the Source Code of the final software product are identical.

f. Perform an analysis to identify any differences between the executable object code used for simulation and the Executable Object Code of the final software product. These differences should be justified and a rationale provided for why they are acceptable.

g. Ensure that the simulation cases and procedures are developed and reviewed according to the guidance in section 23.2.8.3.

For Hardware/Software integration testing (ED-12B / DO-178B section 6.4.3.a), the test procedures should be executed with the Executable Object Code of the final Software product loaded into the target hardware and therefore no credit can be taken for the use of simulation instead of Hardware / Software integration testing.

23.2.8.3

Considerations on simulation cases, procedures and results

For software level A, B and C, in order to gain any certification credit as described in 23.2.8.1 and 23.2.8.2, the simulation cases and procedures should be reviewed against the higher-level requirements. In particular, the applicant should perform analyses to verify that:

- The simulation cases are correct.

- The simulation cases satisfy the criteria of normal range and robustness as defined in section 6.4.2.of ED-12B / DO-178B.

- Simulation cases exist for each requirement that is intended to be verified by simulation.

- The simulation cases were accurately developed into simulation procedures and expected results.

- The simulation results are correct and that discrepancies between actual and expected results are explained.

The same cases and procedures that have been formally reviewed and corrected may be later re-used, if desired, as the basis for the test cases and procedures that are used to test the Executable Object Code against the higher-level requirements (cf. sections 23.2.4.4, 23.2.5.3 and 23.2.6.5).