Methods used for ISVV Process Management are not different from project management methods in general and will not be further described in this document.
4.0 ISVV level definition
4.1 Activity Overview
The objective of the ISVV level definition5 task is to limit the scope and guide subsequent verification and validation activities as well as the methods proposed to be used to perform them. As can be seen from Figure 6, it is a management activity of the ISVV process.
Figure 6 ISVV level definition in context
ISVV level definition (MAN.VV) consists of four tasks as shown in Figure 7 below:
– System level ISVV level definition
– Software technical specification ISVV level definition – Software design ISVV level definition
– Software code ISVV level definition
The figure also shows the inputs and outputs of each task.
5 Activity formerly called ‘Criticality analysis’ but re-named now for clarity reasons to avoid confusion with Safety and Dependability analysis activities.
MAN. Management
IVA. Independent Validation IVE. Independent Verification IVE.TA.Technical Specification Analysis
IVE.DA.Design Analysis IVE.CA.Code Analysis MAN.VV. ISVV level definition
IVA.Validation
MAN.PM.ISVV Process Management
ISVV level definition
Figure 7 ISVV level definition tasks6
6 Note that figure shows only the most important inputs and outputs.
The ISVV Level is a number on an ordinal scale assigned to a system function, software requirement, component or unit to designate the required level of verification and validation.
The following ISVV Levels are defined:
Level Description
ISVVL 0 No ISVV activities are required.
ISVVL 1 Basic ISVV is required.
ISVVL 2 Full ISVV is required.
Table 2: ISVV levels
If ISVV is to be limited to only a subset of the verification and validation activities, some of the ISVV level definition tasks may not be included. For example, for the verification activities, if Code Analysis is not to be carried out, there is no need to carry out Software Code ISVV level definition. If both Design Analysis and Code Analysis are to be excluded, both corresponding ISVV level definition activities may be excluded as well.
The System Level ISVV level definition and the Software Technical Specification ISVV level definition will always have to be carried out - also in the case where ISVV consists only of Independent Validation. In the case where earlier verification activities have been left out (e.g.
there is no Technical Specification Analysis or Design Analysis), but there are still verification activities included in ISVV, ISVV level definition of left out verification activities may still have to be carried out to ensure that prerequisites for the remaining ISVV activity/activities are fulfilled.
For the ISVV level definition both the Software Criticality Analysis and the analysis of other
‘error prone’ factors are needed. It is important for the ISVV supplier to ensure proper scope of the ISVV activities versus planned time and budget. The Software Criticality Analysis is carried out using (Software) Failure Modes, Effects and Criticality Analysis ((S)FMECA), supported by traceability analysis, control flow/call graphs analysis, and complexity measurements.
It is important to emphasise that the use of these methods would not be as rigorous for the purpose of defining the ISVV level as for the Safety and Dependability Analyses to be carried out as part of the development activities. The purpose here is to scope the verification and validation activities, not to find all potential problems (hazards, failures, etc). Also, the performance of these specific analyses depends to some degree on what analyses are already available from the software developer or system integrator.
The main outputs of the ISVV level definition activity are:
– Critical system functions list – Software criticality schema
– Critical software requirements list – Critical software components list – Critical software units list – Error potential questionnaire – ISVV level definition
Items of the lists will usually be grouped by software product. For a given software product, the list will include all items included in the product with a software criticality category and an ISVV level assigned to each of them.
Annex D discusses important concepts of ISVV level definition. Each project shall tailor the applicability of the different tasks and activities to the different software products, components, etc, following Annex D for the ISVV level definition. The intention is to have full and rigorous
ISVV to software parts assigned ISVV Level 2, reducing the number of activities and the rigor while the ISVV Level decreases. All independent verification and validation activities presented in chapters 5.0 to 8.0.are already pre-tailored for different ISVV levels (ISVV level 1 and/or ISVV level 2). This pre-tailoring is presented only as guidance material. In particular, focus shall be put in identifying which technique and method is required to be used for the different activities (often different methods are listed for each activity and not all of them shall be used).
The more rigorous is the method the more rigorous will be the ISVV findings, but it will increase the effort needed.
4.2 Activity Inputs and Prerequisites
The input work products are shown in above Figure 7 with the ISVV level definition tasks.
The ISVV level definition activity is split into four tasks with different starting points in time. A prerequisite for starting any of these activities is the availability of the required input at a satisfactory level of maturity. Please refer to the individual tasks for a more detailed view.
4.3 Activity Outputs
The output work products are shown in above Figure 7 with the ISVV level definition tasks.
4.4 Activity Management
4.4.1 Initiating and Terminating Events
The four tasks of ISVV level definition will normally be carried out prior to the corresponding verification or validation activity. The initiating event of each verification/validation activity thus may be seen also as the initiating event of the ISVV level definition that defines the scope of the activity.
The initial ISVV level definition (System Level ISVV level definition) will normally be carried out by the ISVV customer (and reviewed by the ISVV supplier at the tendering process) as it is an important input for the cost estimation of the ISVV project.
4.4.2 Completion Criteria
The outputs of each of the ISVV level definition tasks shall be reviewed in a joint review meeting between the ISVV supplier and the ISVV customer to determine whether the output provides a sufficient basis for the execution of subsequent verification and validation activities.
4.4.3 Relations to Other Activities
The primary relation of the ISVV level definition to other activities is the Verification and Validation activities, which uses the output of the ISVV level definition to limit scope and guide the performance of the different analyses.
Input to the ISVV level definition activity comes from System and Software Engineering activities as well as from Independent Verification activities previously carried out (ISVV findings).
The initial ISVV level definition (System Level ISVV level definition) is also an important input to the cost estimation task of ISVV management.
These ISVV level definition activities will normally not provide any feedback to the System or Software Engineering activities, they are only to be used to scope the ISVV activities.
4.5 Task Descriptions
4.5.1 System Level ISVV level definition
TASK DESCRIPTION
Title: System Level ISVV level definition Task ID: MAN.VV.T1
Activity: MAN.VV – ISVV level definition Start event: SRR – System Requirements Review End event: PDR – Preliminary Design Review
Responsible: The System Level ISVV level definition shall be carried out by the ISVV customer. The result of the analysis will be reviewed by the ISVV supplier during the tendering process.
Objectives: Identify the ISVV level Inputs:
- From ISVV Customer:
- Software criticality scheme [from Software Engineering]
- Critical function list with criticality categories assigned [from System Engineering]
- Mission and system requirements specification [from System Engineering]
- System architecture [from System Engineering]
- Requirements Baseline [from System Engineering]
- System safety/dependability analyses [from System Engineering]
Sub Tasks (Procedure):
- MAN.VV.T1.S1: Identify the software criticality scheme used for the mission.
- MAN.VV.T1.S2: Evaluate whether the defined software criticality scheme is relevant for the ISVV objective. If it is not, then define a new software criticality scheme for ISVV.
- MAN.VV.T1.S3: If there is a Critical Function List and the criticality scheme it is based on is relevant for the ISVV objective, then use this CFL.
- MAN.VV.T1.S4: If there is no Critical Function List or the ISVV objective does not match the criteria used to derive it, perform a simplified system FMECA along the lines described in Annex E.3.
- MAN.VV.T1.S5: Identify each software product and its supplier. Fill in the error potential questionnaire (see Annex D) for each software product (by the Error potential assessment described in E.2).
- MAN.VV.T1.S6: Assign ISVV level to each system function based on the software criticality category of the function and error potential.
Outputs:
- Critical system functions list - Error potential questionnaires - Software criticality scheme - ISVV level definition
4.5.2 Software Technical Specification ISVV level definition
TASK DESCRIPTION
Title: Software Technical Specification ISVV level definition Task ID: MAN.VV.T2 Activity: MAN.VV – ISVV level definition
Start event: PDR – Preliminary Design Review
End event: TAR – Technical Specification Analysis Review
Responsible: This task will be performed by the ISVV customer or the ISVV supplier as determined by the project contract.
If carried out by the ISVV supplier, the results shall be reviewed and approved by the ISVV customer.
Objectives:
- Define ISVV level
Inputs:
- From ISVV Customer:
- Technical Specification including Interface Control Documents [from Software Engineering]
- Software safety/dependability analyses based on Technical Specification (if existent) [from Software PA]
- Critical system functions list [from System Level ISVV level definition – MAN.VV.T1]
- Error potential questionnaires [from System Level ISVV level definition – MAN.VV.T1]
- Software criticality scheme [from System Level ISVV level definition – MAN.VV.T1]
Sub Tasks (Procedure):
- MAN.VV.T2.S1: For each software product implementing critical system functions, identify any SFMECA based on the Technical Specification available.
- MAN.VV.T2.S2: If an SFMECA exists and the criticality scheme used as a basis is relevant for the ISVV objective, then it may be used as a basis for deriving the critical software requirements list.
- MAN.VV.T2.S3: If no such analyses have been carried out, the quality is too poor, or the ISVV objective differs from the presumptions of the SFMECA, perform a simplified SFMECA based on the Technical Specification including Interface Control Documents. Another simplified way of doing this step is described in Annex E.3 (by Simplified SFMECA).
- MAN.VV.T2.S4: Verify the consistency of the SFMECA with the Critical systems function list. If discrepancies are found, notify the ISVV customer who will have to consider consequences in terms of re-analysis.
- MAN.VV.T2.S5: For each software requirement, derive the software criticality category by identifying the highest criticality category of any failure mode associated with it.
- MAN.VV.T2.S6: Assign an ISVV level to each software requirement based on the software criticality category of the requirement and error potential (there is no need to reassess error potential unless different answers to the error potential questionnaire are expected at this level).
Outputs:
- Critical system functions list (update) - Critical software requirements list - Error potential questionnaire - ISVV level definition
4.5.3 Software Design ISVV level definition
TASK DESCRIPTION
Title: Software Design ISVV level definition Task ID: MAN.VV.T3 Activity: MAN.VV – ISVV level definition
Start event: PDR – Preliminary Design Review End event: TAR – Design Analysis Review
Responsible: This task will be performed by the ISVV supplier and the result reviewed and approved by the ISVV customer.
Objectives:
- Define ISVV level Inputs:
- From ISVV Customer:
- Technical Specification including Interface Control Documents [from Software Engineering]
- Design Definition File: Software architectural design and traceability matrices [from Software Engineering]
- Design Definition File: Software detailed design and traceability matrices (optional) [from Software Engineering]
- Software safety/dependability analyses based on software architectural design or software detailed design (if existent) [from Software PA].
- From ISVV Supplier:
- Critical system functions list [from System Level ISVV level definition – MAN.VV.T1]
- Error potential questionnaires [from Software Technical Specification ISVV level definition – MAN.VV.T2]
- Software criticality scheme [from System Level ISVV level definition – MAN.VV.T1]
- Critical software requirements list [from Software Technical Specification ISVV level definition – MAN.VV.T2]
- Software safety/dependability analysis based on Technical Specification (if existent) [from Technical Specification ISVV level definition – MAN.VV.T2]
- ISVV Findings [from Technical Specification Analysis – IVE.TA]
Sub Tasks (Procedure):
- MAN.VV.T3.S1: Review the findings of and the safety/dependability analysis performed as part of the Technical Specification Analysis. Evaluate the consistency with the critical function list and the critical software requirements list produced by the preceding Criticality Analyses. If discrepancies are found, notify the ISVV customer who will have to consider consequences in terms of re-analysis.
- MAN.VV.T3.S2: If design level safety and dependability analyses exist from the developer, investigate whether these may be used to assign software criticality categories to design components. The software criticality scheme should be relevant for ISVV, the analysis should be based on the same versions of documents as ISVV (or else a delta analysis must be carried out), and the results of any higher level analyses it is based on should not be in conflict with the results of the Technical Specification Analysis.
- MAN.VV.T3.S3: If not, trace the software requirements to software architectural design components. Assign to each software component the highest software criticality category of any requirement tracing to it (by Inspection of traceability matrices).
- MAN.VV.T3.S4: Alternatively, extend the SFMECA carried out at software requirements level by identifying software components as causes for requirements failure modes. This creates an alternative trace from requirements to design components. Assign to each software component the highest software criticality category of any failure mode to which it may contribute (by Simplified SFMECA).
- MAN.VV.T3.S5: Identify any dependency mechanisms for the design language used (e.g. use or call relationships).
- MAN.VV.T3.S6: Analyse the dependency of critical components on other components and adjust the software criticality category of these components to be the same as the critical component depending on them. Some components may be used by several critical components. For these, assign the highest criticality category of any dependent component (by Inspection or Modelling7).
- MAN.VV.T3.S7: Assign an ISVV level to each software component based on the software criticality category of the component and error potential (there is no need to reassess error potential unless different answers to the error potential questionnaire are expected at this level).
- MAN.VV.T3.S8: Software criticality categories and ISVV levels may also be assigned to detailed design software components. The benefit of going to this level of detail for the ISVV level definition should be balanced by the costs induced.
Outputs:
- Critical system functions list (update) - Critical software requirements list (update) - Critical software component list
- Error potential questionnaire - ISVV level definition
4.5.4 Software Code ISVV level definition
TASK DESCRIPTION
Title: Software Code ISVV level definition Task ID: MAN.VV.T4
Activity: MAN.VV – ISVV level definition Start event: CDR – Critical Design Review End event: CAR – Code Analysis Review
Responsible: This task will be performed by the ISVV supplier and the result reviewed and approved by the ISVV customer.
Objectives:
Define ISVV level Inputs:
- From ISVV Customer:
- Design Definition File: Software architectural design and traceability matrices [from Software Engineering]
- Design Definition File: Software detailed design and traceability matrices (optional) [from Software
7 Note that the objective here is not necessarily to build a separate representation of the software design, but to use existing models to understand dependencies between software components
Engineering]
- Design Definition File: Software code [from Software Engineering]
- Software safety/dependability analyses based on code (if existent) [from Software PA]
- From ISVV Supplier:
- Critical system functions list [from System Level ISVV level definition – MAN.VV.T1]
- Error potential questionnaires [from Software Design ISVV level definition – MAN.VV.T3]
- Software criticality scheme [from System Level ISVV level definition – MAN.VV.T1]
- Critical software requirements list [from Software Technical Specification ISVV level definition – MAN.VV.T2]
- Critical software components list [from Software Design ISVV level definition – MAN.VV.T3]
- Software safety/dependability analysis based on Design Definition (if existent) [from Software Design ISVV level definition – MAN.VV.T3]
- ISVV Findings [from Design Analysis – IVE.DA]
Sub Tasks (Procedure):
- MAN.VV.T4.S1: Review the findings of and the safety/dependability analysis performed as part of the Design Analysis.
Evaluate the consistency with the critical system function list, the critical software requirements list and the critical software component list produced by earlier criticality analyses. If discrepancies are found, notify the ISVV customer who will have to consider consequences in terms of re-analysis.
- MAN.VV.T4.S2: If code level safety and dependability analyses exist from the developer, investigate whether these may be used to assign software criticality categories to software units. The software criticality scheme should be relevant for ISVV, the analysis should be based on the same versions of code as ISVV (or else a delta analysis must be carried out), and the results of any higher level analyses it is based on should not be in conflict with the results of the Design Analysis.
- MAN.VV.T4.S3: If not, identify mapping rules from software design components to software units. For each software component (either architectural design component or detailed design component) trace the software component to source code. Assign to each software unit the software criticality category of the software component it implements (by Inspection of traceability matrices).
- MAN.VV.T4.S4: When software complexity is a risky matter, define complexity measure for the software units and calculate the complexity measurement (Software metrics analysis). The complexity measure could be e.g. based on cyclomatic complexity of procedures contained in the unit as well as the number of other units using this unit. Define a threshold to distinguish non-complex from complex units. Note: The metric and threshold are to be agreed between customer and ISVV supplier.
- .MAN.VV.T4.S5: Fill in the error potential questionnaire (see Annex D) for each software unit, taking into account the complexity measures when applicable.
- MAN.VV.T4.S6: Assign ISVV level to each software unit based on the software criticality category of the software unit and error potential (by Error potential assessment).
Outputs:
- Critical system functions list (update) - Critical software requirements list (update) - Critical software component list (update) - Critical software unit list
- Error potential questionnaire - ISVV level definition
5.0 Technical Specification Analysis