ESTEC
ESA ISVV Guide Issue 2.0 29UIDE FOR
NDEPENDENT
OFTWARE
ERIFICATION
ALIDATION
prepared by/préparé par
ESA property.reference/réference
ESA ISVV Guideissue/édition
2revision/révision
0date of issue/date d’édition
December 29, 2008status/état
-Document type/type de document
Technical NoteDisclaimer:
This ISVV Guide is the second issue of the document. The ISVV Guide is provided as is: the European Space Agency gives no warranty nor guarantees whatsoever as to its completeness, adequacy or suitability and shall not be held liable for any direct, indirect nor consequential damages. Use of the ISVV Guide by Readers/Users is made fully at the latter's own risk. Reproduction of all or part of this ISVV Guide is authorised providing the source is referenced.
Feedback:
Readers/users of this guide are invited to provide comments to ESA on eventual inconsistencies in the guide or suggestions for its improvements.
A P P R O V A L
Title
titre
ESA Guide for Independent Software Verification and
Validation
issue
issue
2
revision
revision
0author
auteur
Maria HernekSabine Krueger
date
date
December 29, 2008
approved by
approuvé by
Kjeld Hjortnæs
date
date
C H A N G E
L O G
reason for change /raison du changement
Issue/issue
revision/revision
date/date
2 0 December 29,
2008
C H A N G E
R E C O R D
Issue: 2 Revision: 0
reason for change/raison du changement
page(s)/page(s)
paragraph(s)/paragraph(s)
Overall improvement of document to optimise the information and introduce results from R&D and ISVV process optimisation activities.
contents
:
1.0
Introduction ... 1
1.1
Background and Motivation ... 1
1.2
Purpose ... 1
1.3
Outline ... 1
2.0
What is Independent Software Verification and Validation? ... 2
2.1
Objectives of ISVV ... 2
2.2
ISVV Process Overview ... 3
2.3
Definition of Scope and Budgeting ... 7
2.4
Non-Disclosure and Security ... 8
2.5
Roles and Responsibilities ... 8
2.5.1 ISVV supplier... 8
2.5.2 ISVV Customer... 9
2.5.3 Other roles... 10
2.6
Types of Independence ... 10
3.0
ISVV Process Management ... 12
3.1
Activity Overview ... 12
3.2
Activity Inputs and Prerequisites ... 13
3.2.1 ISVV level definition ... 13
3.2.2 Documents and Code from Software Development ... 13
3.3
Activity Outputs ... 13
3.4
Activity Management ... 13
3.4.1 Initiating and Terminating Events ... 13
3.4.2 Completion Criteria... 14
3.4.3 Relations to other Activities ... 14
3.5
Task Descriptions ... 14
3.5.1 ISVV Process Planning ... 14
3.5.2 ISVV Process Execution, Monitoring and Control. ... 15
3.6
Methods... 16
4.0
ISVV level definition ... 17
4.1
Activity Overview ... 17
4.2
Activity Inputs and Prerequisites ... 20
4.3
Activity Outputs ... 20
4.4
Activity Management ... 20
4.4.1 Initiating and Terminating Events ... 20
4.4.2 Completion Criteria... 20
4.4.3 Relations to Other Activities ... 20
4.5
Task Descriptions ... 21
4.5.1 System Level ISVV level definition... 21
4.5.2 Software Technical Specification ISVV level definition ... 21
4.5.3 Software Design ISVV level definition ... 22
4.5.4 Software Code ISVV level definition... 23
5.0
Technical Specification Analysis ... 25
5.1
Activity Overview ... 25
5.1.1 Software requirements verification ... 26
5.3
Activity Outputs ... 27
5.4
Activity Management ... 27
5.4.1 Initiating and Terminating Events ... 27
5.4.2 Completion Criteria... 27
5.4.3 Relations to other Activities ... 27
5.5
Task Descriptions ... 28
5.5.1 Software Requirements Verification ... 28
6.0
Design Analysis... 30
6.1
Activity Overview ... 30
6.1.1 Architectural Design Independent Verification ... 31
6.1.2 Software Detailed Design Independent Verification ... 32
6.1.3 Software User Manual Analysis ... 33
6.2
Activity Inputs and Prerequisites ... 33
6.3
Activity Outputs ... 34
6.4
Activity Management ... 34
6.4.1 Initiating and Terminating Events ... 34
6.4.2 Completion Criteria... 34
6.4.3 Relations to other Activities ... 34
6.5
Tasks Description ... 34
6.5.1 Architectural Design Verification ... 34
6.5.2 Detailed Design Verification ... 38
6.5.3 Software User Manual Verification ... 43
7.0
Code Analysis... 44
7.1
Activity Overview ... 44
7.1.1 Source Code Verification... 45
7.1.2 Integration and Unit Test Specification and Data Verification ... 46
7.2
Activity Inputs and Prerequisites ... 47
7.3
Activity Outputs ... 47
7.4
Activity Management ... 47
7.4.1 Initiating and Terminating Events ... 47
7.4.2 Completion Criteria... 47
7.4.3 Relations to other Activities ... 48
7.5
Tasks Description ... 49
7.5.1 Source Code Verification... 49
7.5.2 Integration Test Specification and Test Data Verification... 51
7.5.3 Unit Test Procedures and Test Data Verification ... 53
8.0
Independent Validation ... 55
8.1
Activity Overview ... 55
8.1.1 Identification of Test Cases ... 56
8.1.2 Construction of Test Procedures... 58
8.1.3 Execution of Test Procedures ... 59
8.2
Activity Input and Prerequisites ... 60
8.3
Activity Outputs ... 60
8.4
Process Management... 61
8.4.1 Initiating and Terminating Events ... 61
8.4.2 Completion Criteria... 61
8.4.3 Relations to other Activities ... 61
8.5
Task Descriptions ... 61
8.5.1 Identification of Test Cases ... 61
8.5.3 Execution of Test Procedures ... 63
Annex A. Definitions and acronyms... 65
A.1. Definitions... 65
A.2. Acronyms... 68
Annex B. ISVV activity outputs... 70
B.1. ISVV Plan Outline... 70
B.2. Requests for Clarification... 73
B.3. ISVV Report (with ISVV Findings) ... 74
B.4. Progress Reports... 74
B.5. ISVV Findings Resolution Report ... 74
Annex C. Review Item Discrepancy Form Example... 75
Annex D. Summary of ISVV tasks, activities and methods and techniques... 77
Annex E. ISVV Levels and Software Criticality Categories ... 83
E.1. ISVV Level definition overview ... 83
E.2. Error Potential Questionnaire ... 85
E.3. Procedures for Performing Simplified FMECA... 86
Annex F. Methods ... 88
F.1. Formal Methods... 88
F.2. Inspection ... 88
F.3. Modelling ... 89
F.4. Data Flow Analysis ... 91
F.5. Control Flow Analysis ... 91
F.6. Real-Time Properties Verification ... 91
F.7. Reverse Engineering ... 93
F.8. Simulation (Design execution) ... 93
F.9. Software Failure Modes, Effects and Criticality Analysis (SFMECA) ... 93
F.10. Static Code Analysis ... 94
F.11. Traceability Analysis ... 94
Annex G. Checklists ... 95
G.1. Requirements Review Checklists... 95
G.2. Architectural Design Review Checklist ... 98
G.3. Detailed Design Review Checklist... 101
G.4. Software User Manual Review Checklist ... 105
G.5. Code Inspection Checklist... 105
G.6. Unit and Integration Test Review Checklist... 108
G.7. Validation Checklist ... 109
G.8. Model conformance with applicable standards... 109
Annex H. Software Validation Facility ... 110
figures:
Figure 1 ISVV Process Activities ... 3
Figure 2 ISVV tasks in parallel to SW supplier review milestones ... 5
Figure 3 ISVV tasks after SW supplier’s review milestones ... 6
Figure 4 ISVV process management in context ... 12
Figure 5 ISVV Process Management Tasks ... 12
Figure 6 ISVV level definition in context ... 17
Figure 7 ISVV level definition tasks ... 18
Figure 8 Technical Specification Analysis in context ... 25
Figure 9 Technical Specification Analysis activity... 26
Figure 10
Software Requirements Independent Verification ... 26
Figure 11
Design Analysis in context ... 30
Figure 12
Software design analysis... 31
Figure 13
Software Architectural Design Independent Verification... 32
Figure 14
Software Detailed Design Independent Verification ... 33
Figure 15
Software User Manual Independent Verification ... 33
Figure 16
Code Analysis in context ... 44
Figure 17
Code Analysis ... 45
Figure 18
Software Source Code Independent Verification... 46
Figure 19
Integration/Unit Test Procedures and Test Data Verification... 47
Figure 20
Independent Validation in context ... 55
Figure 21
Independent Software Validation ... 55
Figure 22
Subtasks to "Identification of Test Cases" ... 56
Figure 23
Subtasks to "Construction of Test Procedures"... 58
Figure 24
Subtasks to "Execution of Test Procedures" ... 59
tables:
Table 1: Competence requirements for ISVV personnel ... 9Table 2: ISVV levels... 19
Table 3: Dependency between ISVV level, input and analysis ... 57
Table 4: Test case steps ... 58
Table 5: RID Form... 75
Table 6: RID Problem Type Categories ... 76
Table 7: RID Severity Classes ... 76
Table 8: ISVV levels... 83
Table 9: Matrix to derive ISVV level from Software Criticality Category and Error Potential... 84
Table 10: Error Potential Questionnaire... 85
Table 11: Mapping from error potential score to error potential level... 85
Foreword
This ISVV Guide is the result of work carried out for the European Space Agency by the contribution of different companies. Most of the material presented in this Guidebook has been based directly or indirectly on a variety of sources (ESA, European space industry experiences and projects, technical literature sources). These sources as well as the contributors are too numerous to list here, nevertheless we would like to provide acknowledgment to this invaluable contribution from all participants who through different means (direct interviews, the ISVV e-mail address, specific R&D projects, etc) supported the improvement of the ISVV process and this document.
1.0 Introduction
1.1
Background and Motivation
Independent Software Verification and Validation (ISVV) is an engineering practice intended to improve quality and reduce costs of a software product. It is also intended to reduce development risks by having an organisation independent of the software developer’s perform verification and validation of the specifications and code of a software product.
The global objective of this guide is to aid in establishing an improved and coherent ISVV process across the European space industry in a consolidation of existing practice. Special emphasis is placed on process efficiency while the tasks are complementary to the nominal SW supplier’s verification and validation tasks. It is hoped that the guide will also be found useful in other industries (e.g. Automotive, telecom, railway, etc) where software is a component of safety and dependability critical systems.
The guide defines the ISVV process with management, verification, and validation activities. It provides advice on ISVV roles, responsibilities, planning, and communication as well as methods to use for the various verification and validation tasks.
1.2 Purpose
The purpose of this guide is to:
• Define a uniform, cost effective and reproducible ISVV process across projects, and to guide in adapting it to each specific project;
• Assist the industry in getting predictable cost and quality out of the ISVV process; • Clarify the benefits of applying ISVV;
• Improve ISVV project execution by highlighting the many different issues that need to be clarified and considered in the various phases of the project;
• Disseminate best practices with respect to recommended methods for the different verification and validation activities;
• Present a summary of the required capabilities of the Independent SVF in preparation of the development and utilization of a specific one for each project.
The assumed readership of the ISVV Guide is primarily the customers and suppliers of ISVV services, but also software developers, system suppliers (primes) and system customers are likely to find the guide useful, be they verification / validation personnel, quality assurance managers or technical managers.
The guide should be used in the preparation of a request for quotation for an ISVV service, in preparation for a bid, during planning, execution and re-planning of an ISVV project.
1.3 Outline
The document consists of the following sections:
– The introduction of which this outline is a part describes the background and motivation for ISVV as well as the purpose of the ISVV guide.
– Section 2.0 elaborates on the topic of ISVV, describing types of independence, the objectives of ISVV as well as its relationship to development verification and validation. It also provides an overall view of the ISVV process.
– Section 3.0 describes the ISVV process management activity, detailing on ISVV roles, responsibilities, tasks and other aspects of management.
– Section 4.0 describes the ISVV level definition activity, and how it can be used to identify the scope of ISVV.
– Sections 5.0, 6.0, 7.0 describe the verification activities of Technical Specification Analysis, Design Analysis, and Code Analysis respectively.
– Section 8.0 describes the Independent Validation Activity.
– Finally, there are a number of annexes providing more detailed information related to the various ISVV activities.
2.0
What is Independent Software Verification and Validation?
2.1 Objectives
of
ISVV
As with any verification and validation activity, the objective of ISVV is to find faults and to raise confidence in the software subject to the ISVV process. The emphasis of either of these objectives may vary, depending on the maturity of the software, budget, time, the maturity of the software supplier, as well as the distribution of responsibility between the software developer’s V&V and the ISVV supplier’s V&V.
Raising the confidence is particularly important for critical software, whose failure may lead to hazardous events, loss of life and exceptional costs, damage to health, environmental damage, grave economic losses, or loss of reputation. ISVV is therefore usually targeted to find faults in critical and/or safety or dependability related components (including resilience – recovery capabilities). This is also the main emphasis of this guide. However, in other cases, ISVV may target other quality attributes, including security, reusability, and usability.
ISVV should provide added value over the verification and validation carried out by the software developer. The approach of the ISVV supplier thus has to be complementary, introduced by having different:
– organisational missions and values, – objectives,
– processes, – methods, – tools, – people.
The ISVV supplier focuses on finding possible weaknesses and faults, trying to break the software, with a ‘destructive’ attitude.
ISVV shall aim to using methods and tools different from those of the development organisation. In some cases, one method is an alternative to another; in others, methods are complementary and not substitutable.
The ISVV activities shall emphasize on the manual verification and validation activities of the SW supplier (eg. the ISVV supplier could also perform automatic code verifications using different tools than the ones used by the SW supplier). These are the ones that could be better complemented by ISVV.
When performing ISVV, other supporting processes should be performed in parallel to the different ISVV tasks: e.g. Configuration Management process, the documentation process, the SPA process, the Project management process, the risk management process, the problem resolution process.
2.2
ISVV Process Overview
The ISVV Process consists of 6 activities: two management activities, three verification activities, and one validation activity. The activities are shown in the figure below:
Figure 1 ISVV Process Activities
ISVV Process Management (MAN.PM) is concerned with issues such as roles, responsibilities, planning, budgeting, communication, competence, confidentiality etc. It involves responsibilities of both the ISVV customer and the ISVV supplier.
ISVV level definition (MAN.VV) is an activity supporting both ISVV Process Management and the Verification and Validation tasks. It provides important input for ISVV planning and how the available budget can be best utilized. The objective of the ISVV level definition task is to limit the scope and guide subsequent verification and validation activities as well as the methods proposed to be used to perform them.
Technical Specification Analysis (IVE.TA) is verification of the Technical Specification, i.e. software requirements. The activity ends with a Technical Specification Analysis Review (TAR).
Design Analysis (IVE.DA) is verification of the Software Architectural Design and the Software Detailed Design. The activity ends with a Design Analysis Review (DAR).
Code Analysis (IVE.CA) is verification of the software source code. The activity ends with a Code Analysis Review (CAR).
Validation (IVA) is testing of the software to demonstrate that the implementation meets the Technical Specification in a consistent, complete, efficient and robust way. The activity ends with an Independent Validation Review (IVR).
Figure 2 and Figure 3 relate the ISVV activities to the software development processes and the review milestones defined by [ECSS-E-40B:2003]. Two different life cycle process dependencies are shown: a) when the ISVV activities are performed in parallel to the review milestones, and b) when they are performed after the review milestones. The four ISVV reviews are indicated in the figures to highlight the connection with the other milestones. The figures indicate possible early and likely start times as well as end times of the activities. More specific guidance is provided with the individual activity.
Scheduling the ISVV project has a strong dependence on the progress of the software development projects. Delays in software development activities will cause corresponding delays in ISVV activities1.
1
A scheduled date for the start of activities may be provided, with the understanding that this date may have to change if deliverables from the software development projects are late.
MAN. Management
IVA. Independent Validation IVE. Independent Verification IVE.TA.Technical Specification Analysis
IVE.DA.Design Analysis IVE.CA.Code Analysis MAN.VV. ISVV level definition
IVA.Validation
ISVV suppliers shall follow the same life cycle as the SW supplier: e.g. supplier’s iterative or incremental life cycle imply iterative or incremental ISVV tasks; or supplier’s joint architectural design with detailed design, affects ISVV activities in similar way. Repetition of ISVV tasks as well as verification of new or largely modified parts of products should be carefully planned beforehand in cooperation with ISVV customer to ensure most coverage as well as highest efficiency. But not all correspondent ISVV tasks have to be performed when following de same life cycle. Examples of this might include: a) single ISVV tasks to be performed after a specific milestone, etc.
This guide does not identify specific initiating events for ISVV activities. A general recommendation is that input documents and code from the software suppliers should be sufficiently mature. This is normally achieved at the delivery of review data package or after the implementation of project reviews: PDR, DDR, and CDR. The Independent Validation activity may already start with the identification of test cases as early as PDR, when stable documentation becomes available. To carry out the independent software validation testing effectively requires completion of the software development validation testing against the TS – this usually finishes at CDR. Note: MAN.VV activities are drawn as VV in the figures.
ISVV projects should aim at providing all ISVV findings before SW-QR, but this is to be defined per project.
In some cases, documents may be mature earlier, or it may be desirable to have ISVV providing input to the development review – along with other review comments. In this case, ISVV activities have to start earlier.
ISVV S o ft w a re D e v e lopme n t Systems Engineering
Delivery and acceptance
VV
VV
SRR PDR DDR CDR QR
Time
AR
TAR DAR CAR IVR
TS Analysis AD Analysis Code Analysis Independent Validation VV DD Analysis SUM analysis Requirements & Architecture Engineering
Design and Implementation Engineering
Validation
Verification
VV
ISVV So ft w a re D e v e lo pme n t Systems Engineering
Delivery and acceptance
VV TS Analysis VV SRR PDR DDR CDR QR Time AR
TAR DAR CAR IVR
VV
Design Analysis VV
Independent Validation Code Analysis
Design and Implementation Engineering
Validation
Verification Requirements &
Architecture Engineering
Figure 3 ISVV tasks after SW supplier’s review milestones
Each of the ISVV activities is described in detail in the following sections. The ISVV Process Management activity is given special treatment, but otherwise the main structure of an activity description is:
– Activity overview
– Activity inputs and prerequisites – Activity outputs
– Activity management
− Initiating and terminating events − Completion criteria
− Relations to other activities – Task descriptions
– Methods
Every activity is broken down into tasks and sometimes sub-tasks. Each task description (with the exception of project management tasks) is described in a table format with the following fields:
Title: Name of the task Task ID: Task identifier
Activity: Identifier and name of the activity
Start Event: Start constraint for the task (might be tailored depending on the
characteristics/objectives of specific ISVV projects)
End Event: End constraint for the task (might be tailored depending on the
Responsible: Identification of the organization responsible for the task execution, the ISVV
supplier or the ISVV customer.
Objectives: Main objectives to be accomplished by the task Inputs: Inputs to the task
Sub Tasks (per ISVV Level): Task breakdown into subtasks, organised per ISVV level. Outputs: Outputs of the task
A specific ISVV project may include all, one, or some of the previously referred verification and validation activities and their associated tasks and subtasks. There are dependencies between the activities; output of previous activities is often used as input for later activities2.
Users of this guide are encouraged to propose “advanced” techniques beyond the ones in the guide providing they can justify their performance.
2.3
Definition of Scope and Budgeting
The budget for ISVV should reflect the criticality of the software to be scrutinised. The higher the criticality of the system (and the software) is, the bigger the effort for the verification and validation would be.
This is the objective of the so-called ISVV level definition (see definition in Annex D), which identifies the ISVV Level of software items at various stages (software requirements, component, unit), both reducing the number of items subject to ISVV and determining which verification and validation tasks to carry out for each individual item.
The ISVV costs are mainly composed by man-hour costs, travelling costs and tool costs. Man-hour costs are usually dominant. Cost drivers are:
• Volume of specifications and code to be analysed. The ISVV Level definition is used to scope volume, so that it fits with available budget.
• The rigour with which an ISVV task is to be carried out. This depends on the ISVV Level.
• Methods to be used. Some methods are more labour intensive than other, but on the other hand they may also increase the value of the results.
• Repetitions in the development cycle. SW development is usually carried out in an iterative way (e.g. Incremental). Typically this means that ISVV activities have to be repeated on different versions of specifications and source code. This may have a big impact on costs, and for fixed-price contracts it is thus crucial that the number of repetitions is defined before the project starts.
• Complexity of the project. E.g., a project with many SW suppliers will require more management hours
Travelling costs depends on whether ISVV supplier staff is required to be present at milestone reviews or not. This needs to be clarified in the Statement Of Work issued by the ISVV customer.
Tools support specific methods for specific verification and validation tasks. The use of tools may greatly increase the efficiency of carrying out ISVV tasks, thereby reducing man-hours.
2
If one or more of the activities are defined to be outside the scope of the ISVV project, some of the tasks of the activity may nevertheless have to be performed for the prerequisites (in terms of required input) of other activities to be fulfilled. The dependencies are described as part of the individual activity descriptions.
2.4
Non-Disclosure and Security
Spacecraft software is high value intellectual property. It is therefore important that access to documents and code (both source and executable code) is strictly controlled when handed over to the ISVV supplier3. The ISVV supplier and other stakeholders involved in the ISVV process must fulfil requirements both with respect to non-disclosure and with respect to secure handling of information.
The ISVV customer must in cooperation with the software suppliers (or the intellectual property owner) and other stakeholders (system supplier/customer) determine the confidentiality requirements, including:
– whether there should be different confidentiality classes for documents and what those classes are;
– requirements for distribution and storage of confidential documents; – requirements for personnel authorised to handle confidential documents.
A Non Disclosure Agreement shall be established among the ISVV supplier, the ISVV customer and the SW supplier in order to preserve the confidentiality of the data.
It is not recommended that the ISVV supplier performs ISVV activities at SW supplier’s site. There are several reasons for this: independence might be jeopardised, increase in travelling cost, difficulties in tool and environmental share, timeframe of analyses may be limited etc. The ISVV customer must also identify the documents required for the ISVV process and their confidentiality class.
The ISVV supplier should have an information security management system in place to ensure that distribution, storage, and handling of data fulfils the confidentiality requirements (e.g. based on [BS 7799-2:2002]).
2.5
Roles and Responsibilities
Independent Software Verification and Validation is a service provided by an ISVV supplier to an ISVV customer. Their roles and responsibilities are described in subsequent sections. 2.5.1 ISVV supplier
Independent software verification and validation requires special competence.
Requirements on the competence on the individual should cover formal education, experience, as well as personal traits. There is still no consensus on what the requirements for ISVV personnel should be, but an example is provided here below:
Formal education ISVV personnel should have a university degree in software
engineering, computer science or similar. It will also be beneficial to have some formal background in hardware electronics and systems engineering as well as the domain itself, e.g. aerospace. Personnel should also have received proper training in quality assurance and quality control and testing.
3
The software developer will most likely require the ISVV supplier not to be involved in any kind of competing software development.
Experience ISVV personnel should have at least 5 years of working experience with software development of which at least 2 should be related to verification, validation or quality assurance. ISVV personnel should also have at last 2 years of experience within the space domain. If models are produced, the ISVV team is required to have a good knowledge of the modelling tool suite – if not proficiency – to fully utilise the verification and simulation capabilities of the tool suite and to correctly interpret the results.
Personal traits ISVV personnel should be “creative destructors”, rigorous, process mindful, objective, should have a critical attitude and be result-oriented. ISVV personnel should also have:
- strong communication skills
- be pragmatic, direct, able to simplify problems - have a critical mindset, able to explore and test, try
- have generic knowledge about the domain environment and current and upcoming technologies
Table 1: Competence requirements for ISVV personnel
The ISVV personnel must thus exhibit creativity at breaking the system, at being “destructive” but respecting and considering the full scope of the mission requirement set. ISVV suppliers should also have the attitude of being the USER of the SW product, e.g.: with the operator's view focusing for example either on successfully controlling the software when recovering from errors or on how the system enters into safe mode.
An ISVV project is usually carried out by an ISVV team, not a single individual. The ISVV team should compose a mix of complementary competencies. The team as such should be familiar with all methods and tools to be employed for the analyses. In addition, the ISVV team manager should be experienced with project management, including the management of ISVV projects. The project manager must also be able to handle the contractual and human relations aspects of the project, and should also have sufficient personal authority to defend the findings of the ISVV team. In addition to what is mentioned in above Table 1, for some ISVV tasks (e.g. for safety and dependability verification, or the Independent validation tasks, etc) someone at the ISVV supplier should have general space system knowledge expertise. The ISVV supplier should have a suitable quality management system (fulfilling the requirements of [ISO 9000:2000]) as well as an information management system).
The ISVV process has many uncertainties (availability of inputs from the software supplier, etc) and risky elements (maturity of the elements under ISVV, maturity of the SVF, etc) that should be properly managed and controlled by the ISVV supplier through a formalised risk management process.
2.5.2 ISVV Customer
The ISVV customer has the following responsibilities: − Define the ISVV objectives
− Perform the initial ISVV level definition − Define the ISVV scope and budget
− Approve the ISVV supplier’s ISVV Plan and control its implementation − Ensure NDAs are signed up between ISVV suppliers and SW suppliers − Filter ISVV findings
In European space projects, the ISVV customer has traditionally been either the prime or the end customer. The prime should not be the ISVV customer if the prime itself (or any of its subsidiaries) is also developing software subject to ISVV.
2.5.3 Other roles
In addition, the ISVV process may have interfaces to other roles: – Software supplier (software developer)
– Software validation facility supplier
– System supplier (system integrator, prime, software customer) – System customer (system owner)
One of the two latter roles is also likely to be the ISVV customer.
2.5.3.1 Responsibilities of Software suppliers
The involvement of the software supplier in ISVV includes:
– Assisting the ISVV customer in responding to requests for clarifications from the ISVV supplier;
– Assisting the ISVV customer in assessing the findings of the ISVV supplier, their criticality and resolution;
– Investigating and following up software problem reports resulting from ISVV findings.
All communication between ISVV supplier and any of the software suppliers (when allowed) shall be copied to the ISVV customer.
2.5.3.2 Interface with Software Validation Facility supplier
The Software Validation Facility supplier is the party providing the Software Validation Facility for the ISVV supplier’s independent validation activity.
The involvement of the SVF supplier could be minimal, i.e. just providing the SVF for a given period, or it could involve tasks such as specification and execution of test procedures, and reporting of test results.
It is the ISVV customer’s responsibility to ensure the ISVV supplier gets (or gets access to) the SVF. The recommendation of this ISVV guide is that the SVF is provided to the ISVV supplier. This secures the ISVV project’s access to the SVF, also in critical phases of the project where resource contention would otherwise easily occur.
2.6
Types of Independence
In the context of ISVV, independence is intended to introduce three important benefits: – Separation of concerns. Any person or organisation is likely to discover that their activity
inevitably produces conflicting demands and interests. Clearly separating roles and responsibilities ensures that such conflicts do not arise and also gives confidence of this to other stakeholders.
– A different view. All persons have a limited horizon of understanding within which texts (both written and oral) are interpreted and produced. A second opinion contributes to complement the view of the other by identifying omissions, ambiguities, factual errors, logic errors etc.
– Effectiveness and productivity in verification and validation activities. Staff specialised in independent software verification and validation develops technical competence and
motivation that should lead to more effective and productive work. This is especially the case with verification and validation methods that necessitate the application of
sophisticated tools.
ISVV implies independence, additional and complementary activities from the SW supplier’s verification and validation.
Many standards (e.g. [IEC 61508-1:1998]) distinguish between:
– independent person, who may belong to the same department as the writer/developer, but should not have been involved in writing the specification or the code
– independent department, that requires verification to be carried out by people from a different department within the same organisation, and
– independent organisation that must be different legal entities with different management groups and preferably different owners
The higher the criticality of the system (and the software – see critical item definition in Annex A.1), the more independence is required.
The IEEE Standard for Software Verification and Validation [IEEE 1012:1998] distinguishes between different types of independence addressing these concerns:
– technical independence, ("fresh viewpoint") by an independent person, is an important method to detect subtle errors overlooked by those too close to the solution. For software tools, technical independence means that the IV&V effort uses or develops its own set of test and analysis tools separate from the developer's tools
– managerial independence, allowed to submit to program management the IV&V results, anomalies, and findings without any restrictions (e.g., without requiring prior approval from the development group) or adverse pressures, direct or indirect, from the development group, and
– financial independence preventing situations where the IV&V effort cannot complete its analysis or test or deliver timely results because funds have been diverted or adverse financial pressures or influences have been exerted.
In European space industry, full technical, managerial and financial independence is required for ISVV of critical software. The ISVV supplier is required to be an organisation independent of the software supplier as well as the prime (system integrator).
3.0
ISVV Process Management
3.1 Activity
Overview
The objective of ISVV Process Management is to define the overall ISVV plan and to control and monitor the ISVV process. As can be seen from Figure 4 it is a management activity of the ISVV process.
Figure 4 ISVV process management in context ISVV Process Management (PM) consists of two tasks as shown in Figure 5: – ISVV Process planning
– ISVV Process monitoring and control
The figure also shows the inputs and outputs of each task.
Figure 5 ISVV Process Management Tasks4
4
Note that figure shows only the most important inputs and outputs.
ISVV process management Software Technical ISVV Process Planning
ISVV Process Monitoring and Control Software Development Plan Software product Assurance Plan Software Verification and Validation Plan
Documents and code (SW development)
ISVV level definition
ISVV Plan
Request for Clarification
ISVV Report (with ISVV findings)
ISVV findings resolution report Progress reports
ISVV Plan
ISVV Report (with ISVV findings) Criticality analyses (System engineering)
MAN. Management
IVA. Independent Validation IVE. Independent Verification IVE.TA.Technical Specification Analysis
IVE.DA.Design Analysis IVE.CA.Code Analysis
IVA.Validation
MAN.PM.ISVV Process Management
3.2
Activity Inputs and Prerequisites
The input work products are shown in above Figure 5 defining the ISVV Process Management Tasks.
There are no particular prerequisites for starting the ISVV Management Activity. 3.2.1 ISVV level definition
The ISVV level definition activity (see Annex E with the definition of the ISVV levels and software criticality categories) produces critical items lists defining the scope for subsequent verification and validation activities. The activities are either carried out by the ISVV customer or the ISVV supplier. In addition to the software criticality categorization it also identifies other factors that influence the ISVV level to be assigned to the software product subject of ISVV. The scope resulting from the ISVV level definition must be reflected in the ISVV plan. The initial ISVV plan is based (among other inputs) on the initial ISVV level definition task results and updated as later ISVV level definition refines the scope.
3.2.2 Documents and Code from Software Development
The main input to the ISVV project is the software documents and code to be verified and validated. Additional documentation may also be required, e.g. system requirements, system architecture, safety analyses, etc. In addition, process documents such as development standards and quality assurance procedures may also be required. Documents and code to be verified or validated should be reasonably mature and stable before being subject to ISVV activities. This normally means that they have been submitted for or been through development reviews. Earlier versions of the documents and code may be provided to the ISVV supplier for familiarisation, especially if deadlines for the actual ISVV are short.
All documentation available from the software development should be delivered to the ISVV supplier when published in order to favour his knowledge acquisition, even when no ISVV activity is due to be performed (eg: RB release, etc). .
Other inputs from the SW supplier to be delivered to the ISVV Customer are needed for proper management of the ISVV activities. These inputs are: SW development plans and schedules, NDAs, managerial constraints like participation or synchronization with reviews, ISVV goals, critical items lists, etc. Then also part of these information shall be transferred/filtered to the ISVV supplier by the ISVV Customer.
To ensure the efficiency of ISVV activity it is important that inputs are delivered and received in electronic searchable original format.
3.3
Activity Outputs
The output work products are shown in above Figure 5 defining the ISVV Process Management Tasks.
3.4 Activity
Management
3.4.1 Initiating and Terminating Events
The ISVV Management activity starts for the ISVV customer when it becomes clear that a software product for which the customer is responsible (either as developer or integrator) will require ISVV. This may be at an early stage in the ISVV customer’s process of bidding for the development of the software or the system containing the software. The ISVV customer starts the activity when preparing a tender package for ISVV services.
ISVV Management ends with the close of the ISVV contract, i.e. with the acceptance by the ISVV customer of all deliverables required by the contract and described in the ISVV Plan. 3.4.2 Completion Criteria
The ISVV Management activity completes when all ISVV activities, tasks and subtasks defined by the ISVV Plan have been carried out and all deliverables have been accepted by the ISVV customer.
3.4.3 Relations to other Activities
The ISVV Management activity shall manage all of the other activities of the ISVV project. The ISVV level definition activities provide important input to the ISVV Management activity for budgeting and planning.
3.5 Task
Descriptions
Note that responsibility for the ISVV Management tasks is shared between the ISVV Customer and the ISVV Supplier. Responsibility is defined at the subtask level.
3.5.1 ISVV Process Planning
TASK DESCRIPTION
Title: ISVV Process Planning Task ID: MAN.PM.T1
Activity: ISVV Management
Start event: Start of project
End event: End of project
Responsible: ISVV Customer and ISVV Supplier
Objectives: Plan the ISVV process
Inputs:
- From ISVV Customer:
- Criticality Analyses (from System Engineering) - ISVV level definition
- Software Development Plan - Software Product Assurance Plan - Software Verification and Validation Plan - From ISVV Supplier:
- ISVV level definition
Sub Tasks (per ISVV Level):
- MAN.PM.T1.S1: Define ISVV objectives (ISVV Customer)
The main objectives of the ISVV (what quality attribute(s) are critical?) must be defined by the ISVV Customer. See also section 2.1.
- MAN.PM.T1.S2: Perform System Level ISVV level definition (ISVV Customer)
The ISVV Customer should perform an analysis to identify the need for ISVV, its scope and the initial critical items list (see section 4.5.1).
- MAN.PM.T1.S3: Define the ISVV scope and determine the ISVV budget (ISVV Customer)
The ISVV Customer should determine the overall ISVV budget frame based on the mission costs, size of SW product (documents and code) and the ISVV scope and level (see sections 4.1).
- MAN.PM.T1.S4: Perform Technical Specification ISVV level definition (ISVV Customer or ISVV Supplier)
Perform the Technical Specification analyse to identify the ISVV scope, level, and critical software requirements list. This may be carried out by the ISVV Customer or the ISVV Supplier. See also section 4.5.2.
- MAN.PM.T1.S5: Estimate ISVV scope and budget (ISVV Supplier)
The ISVV Supplier should do an independent estimation of the ISVV budget. See section 4.1.
- MAN.PM.T1.S6: Develop ISVV plan (ISVV Supplier)
The ISVV Supplier must define an ISVV plan (a draft could be part of the proposal). The plan should be approved by the ISVV Customer. The developer’s software development plan, software product assurance plan, and software verification and validation plan should be taken into account if available (overall coordination planning data is to be provided by the ISVV Customer). An outline of a sample ISVV plan is found in Annex B.1.
- MAN.PM.T1.S7: Approve ISVV Plan (ISVV Customer)
The ISVV Customer should approve the ISVV plan developed by the ISVV Supplier. An outline of a sample ISVV plan is found in Annex B.1.
- MAN.PM.T1.S8: Determine confidentiality issues and prepare NDAs (ISVV Customer)
It is the responsibility of the ISVV Customer to clarify confidentiality requirements and ensure these are kept throughout the project through the signing of Non-Disclosure Agreement with the ISVV Supplier and any of its sub-contractors (see Section 2.4).
- MAN.PM.T1.S9: Approve scope definition resulting from ISVV level definition (ISVV Customer)
All ISVV level definition results must be approved by the ISVV Customer. See also section 3.2.1.
Outputs:
- ISVV plan (ISVV Supplier)
3.5.2 ISVV Process Execution, Monitoring and Control.
TASK DESCRIPTION
Title: ISVV Process monitoring and control Task ID: MAN.PM.T2
Activity: ISVV Management
Start event: Project start
End event: Project end
Responsible: ISVV Customer and ISVV Supplier
Objectives: Execute, monitor, and control the ISVV process
Inputs:
- From ISVV Customer:
- ISVV level definition (from System Engineering) - Software Development Plan
- Software Product Assurance Plan
- Documents and Code from Software Development - ISVV Findings Resolution Report
- From ISVV Supplier:
- ISVV level definition [MAN.VV] - ISVV Plan [MAN.PM.T1]
Sub Tasks (per ISVV Level):
- MAN.PM.T2.S1: Manage ISVV project (ISVV Supplier)
The ISVV Supplier must manage the project in accordance with the ISVV plan. This includes schedule management ( the dependency of SW supplier’s schedule changes), budget management, resource management, activity management, risk management, quality management, document management, and security management.
- MAN.PM.T2.S2: Submit documentation and code to ISVV Supplier (ISVV Customer)
It is the responsibility of the ISVV Customer to provide all documentation and code necessary for ISVV planning and for the verification and validation activities to the ISVV Supplier. See also section 3.2.2.
- MAN.PM.T2.S3: Check received documentation (ISVV Supplier)
Any documentation and code received from the ISVV Customer or other parties of the ISVV should be registered and checked by the ISVV Supplier.
The ISVV Suppliers shall familiarize themselves with the system where the SW is to be working in, supplier’s development environment, the details of the SW product to be ISVV’ed, the software validation facility and tools in which it will be validated, etc.
- MAN.PM.T2.S5: Submit the verification and validation testing environment (ISVV Customer)
Both the development environment and the validation testing environment from the SW supplier shall be available to the ISVV Supplier either delivered by the ISVV Customer or to be acquired by the ISVV supplier .
- MAN.PM.T2.S6: Perform verification and validation activities (ISVV Supplier)
The ISVV Supplier must carry out the verification and validation activities as described in the ISVV plan.
- MAN.PM.T2.S7: Request clarifications (ISVV Supplier)
The ISVV Supplier may request clarification from the ISVV Customer. See Annex B.2.
- MAN.PM.T2.S8: Respond to Requests for Clarification (ISVV Customer)
Whenever the ISVV Supplier issues a Request for Clarification, the ISVV Customer should provide feedback in a timely manner (see Annex B.2)
- MAN.PM.T2.S9: Report early ISVV findings (ISVV Supplier)
The ISVV Supplier may provide early feedback on findings to the ISVV Customer.
- MAN.PM.T2.S10: Review early ISVV Findings (ISVV Customer)
The ISVV Customer shall review received early ISVV findings for criticality and impact on the software/system, and shall take action as appropriate.
- MAN.PM.T2.S11: Produce ISVV report (ISVV Supplier)
For each ISVV activity (as defined by the ISVV plan), the ISVV Supplier must produce an ISVV report in which all of the findings shall be reported and main findings shall be highlighted. See Annex B.3.The ISVV supplier shall perform an internal review of the findings before submission.
- MAN.PM.T2.S12: Filter ISVV findings (ISVV Customer)
The ISVV Customer shall review and filter the ISVV findings in order to optimise their disposition and eventual implementation.
- MAN.PM.T2.S13: Draft disposition of ISVV findings (SW supplier)
The ISVV findings shall be replied by the SW supplier to be later discussed during a review meeting (see below).
- MAN.PM.T2.S14: Conduct Review Meeting (ISVV Customer)
The ISVV findings and their resolution are discussed during a review meeting (or off line discussion meeting) with participation of all related parties. The meeting is the responsibility of the ISVV Customer.
- MAN.PM.T2.S15: Produce ISVV findings resolution report (ISVV Customer)
In response to each ISVV report, the ISVV Customer should produce an ISVV findings resolution report, describing how each finding is resolved. The reports should be distributed to the ISVV Supplier and to the end customer (see section 3.3). in case of discrepancy between ISVV supplier and software provider it is up to the ISVV customer to determine the verdict.
- MAN.PM.T2.S16: Implement resolutions (ISVV Customer)
The ISVV Customer is responsible for ensuring that the resolutions described in the ISVV findings resolution report are implemented. The ISVV Supplier is not responsible for following-up the findings.
- MAN.PM.T2.S17: Update ISVV level definition (ISVV Supplier)
The ISVV level definition is may be updated throughout the project to further limit the scope of subsequent verification and validation activities. This is the responsibility of the ISVV Supplier, although the ISVV Customer may also be involved. See sections 4.5.3 and 4.5.4.
Outputs:
- Requests for Clarification (ISVV Supplier) (Optional) - ISVV Plan (ISVV supplier – update)
- ISVV Report (with ISVV Findings) (ISVV Supplier) - ISVV findings resolution report (ISVV Customer) - Progress Reports (ISVV Supplier)
3.6 Methods
Methods used for ISVV Process Management are not different from project management methods in general and will not be further described in this document.
4.0
ISVV level definition
4.1 Activity
Overview
The objective of the ISVV level definition5 task is to limit the scope and guide subsequent verification and validation activities as well as the methods proposed to be used to perform them. As can be seen from Figure 6, it is a management activity of the ISVV process.
Figure 6 ISVV level definition in context
ISVV level definition (MAN.VV) consists of four tasks as shown in Figure 7 below: – System level ISVV level definition
– Software technical specification ISVV level definition – Software design ISVV level definition
– Software code ISVV level definition
The figure also shows the inputs and outputs of each task.
5
Activity formerly called ‘Criticality analysis’ but re-named now for clarity reasons to avoid confusion with Safety and Dependability analysis activities.
MAN. Management
IVA. Independent Validation IVE. Independent Verification IVE.TA.Technical Specification Analysis
IVE.DA.Design Analysis IVE.CA.Code Analysis
MAN.VV. ISVV level definition
IVA.Validation
ISVV level definition
Software Technical Specification ISVV level definition
Software Design ISVV level definition
Mission and system requirement specification
System Level ISVV level definition
Software code ISVV level definition
System architecture Requirements Baseline System safety/dependability analyses Software criticality scheme Critical system function list Error potential questionnaire Technical specification (including ICDs) Software safety/ dependability analyses
(based on req) [PAF]
ISVV findings (from TS analysis) Critical software requirements list Software architectural design [DDF] Software detailed design [DDF] Software safety/ dependability analyses (based on design) [PAF]
Critical software components list
Software code [DDF]
Software safety/ dependability analyses (based on code) [PAF]
Software criticality scheme Error potential questionnaire Critical software components list
Critical software units list Critical software requirements list ISVV level definition
Critical system function list Critical function list
with criticality categories assigned
ISVV findings (from design analysis)
Figure 7 ISVV level definition tasks6
6
The ISVV Level is a number on an ordinal scale assigned to a system function, software requirement, component or unit to designate the required level of verification and validation. The following ISVV Levels are defined:
Level Description
ISVVL 0 No ISVV activities are required. ISVVL 1 Basic ISVV is required.
ISVVL 2 Full ISVV is required.
Table 2: ISVV levels
If ISVV is to be limited to only a subset of the verification and validation activities, some of the ISVV level definition tasks may not be included. For example, for the verification activities, if Code Analysis is not to be carried out, there is no need to carry out Software Code ISVV level definition. If both Design Analysis and Code Analysis are to be excluded, both corresponding ISVV level definition activities may be excluded as well.
The System Level ISVV level definition and the Software Technical Specification ISVV level definition will always have to be carried out - also in the case where ISVV consists only of Independent Validation. In the case where earlier verification activities have been left out (e.g. there is no Technical Specification Analysis or Design Analysis), but there are still verification activities included in ISVV, ISVV level definition of left out verification activities may still have to be carried out to ensure that prerequisites for the remaining ISVV activity/activities are fulfilled. For the ISVV level definition both the Software Criticality Analysis and the analysis of other ‘error prone’ factors are needed. It is important for the ISVV supplier to ensure proper scope of the ISVV activities versus planned time and budget. The Software Criticality Analysis is carried out using (Software) Failure Modes, Effects and Criticality Analysis ((S)FMECA), supported by traceability analysis, control flow/call graphs analysis, and complexity measurements.
It is important to emphasise that the use of these methods would not be as rigorous for the purpose of defining the ISVV level as for the Safety and Dependability Analyses to be carried out as part of the development activities. The purpose here is to scope the verification and validation activities, not to find all potential problems (hazards, failures, etc). Also, the performance of these specific analyses depends to some degree on what analyses are already available from the software developer or system integrator.
The main outputs of the ISVV level definition activity are: – Critical system functions list
– Software criticality schema
– Critical software requirements list – Critical software components list – Critical software units list – Error potential questionnaire – ISVV level definition
Items of the lists will usually be grouped by software product. For a given software product, the list will include all items included in the product with a software criticality category and an ISVV level assigned to each of them.
Annex D discusses important concepts of ISVV level definition. Each project shall tailor the applicability of the different tasks and activities to the different software products, components, etc, following Annex D for the ISVV level definition. The intention is to have full and rigorous
ISVV to software parts assigned ISVV Level 2, reducing the number of activities and the rigor while the ISVV Level decreases. All independent verification and validation activities presented in chapters 5.0 to 8.0.are already pre-tailored for different ISVV levels (ISVV level 1 and/or ISVV level 2). This pre-tailoring is presented only as guidance material. In particular, focus shall be put in identifying which technique and method is required to be used for the different activities (often different methods are listed for each activity and not all of them shall be used). The more rigorous is the method the more rigorous will be the ISVV findings, but it will increase the effort needed.
4.2
Activity Inputs and Prerequisites
The input work products are shown in above Figure 7 with the ISVV level definition tasks. The ISVV level definition activity is split into four tasks with different starting points in time. A prerequisite for starting any of these activities is the availability of the required input at a satisfactory level of maturity. Please refer to the individual tasks for a more detailed view.
4.3 Activity
Outputs
The output work products are shown in above Figure 7 with the ISVV level definition tasks.
4.4 Activity
Management
4.4.1 Initiating and Terminating Events
The four tasks of ISVV level definition will normally be carried out prior to the corresponding verification or validation activity. The initiating event of each verification/validation activity thus may be seen also as the initiating event of the ISVV level definition that defines the scope of the activity.
The initial ISVV level definition (System Level ISVV level definition) will normally be carried out by the ISVV customer (and reviewed by the ISVV supplier at the tendering process) as it is an important input for the cost estimation of the ISVV project.
4.4.2 Completion Criteria
The outputs of each of the ISVV level definition tasks shall be reviewed in a joint review meeting between the ISVV supplier and the ISVV customer to determine whether the output provides a sufficient basis for the execution of subsequent verification and validation activities. 4.4.3 Relations to Other Activities
The primary relation of the ISVV level definition to other activities is the Verification and Validation activities, which uses the output of the ISVV level definition to limit scope and guide the performance of the different analyses.
Input to the ISVV level definition activity comes from System and Software Engineering activities as well as from Independent Verification activities previously carried out (ISVV findings).
The initial ISVV level definition (System Level ISVV level definition) is also an important input to the cost estimation task of ISVV management.
These ISVV level definition activities will normally not provide any feedback to the System or Software Engineering activities, they are only to be used to scope the ISVV activities.
4.5
Task Descriptions
4.5.1 System Level ISVV level definition
TASK DESCRIPTION
Title: System Level ISVV level definition Task ID: MAN.VV.T1
Activity: MAN.VV – ISVV level definition
Start event: SRR – System Requirements Review
End event: PDR – Preliminary Design Review
Responsible: The System Level ISVV level definition shall be carried out by the ISVV customer. The result of the analysis will be reviewed by the ISVV supplier during the tendering process.
Objectives: Identify the ISVV level
Inputs:
- From ISVV Customer:
- Software criticality scheme [from Software Engineering]
- Critical function list with criticality categories assigned [from System Engineering] - Mission and system requirements specification [from System Engineering] - System architecture [from System Engineering]
- Requirements Baseline [from System Engineering]
- System safety/dependability analyses [from System Engineering]
Sub Tasks (Procedure):
- MAN.VV.T1.S1: Identify the software criticality scheme used for the mission.
- MAN.VV.T1.S2: Evaluate whether the defined software criticality scheme is relevant for the ISVV objective. If it is not,
then define a new software criticality scheme for ISVV.
- MAN.VV.T1.S3: If there is a Critical Function List and the criticality scheme it is based on is relevant for the ISVV
objective, then use this CFL.
- MAN.VV.T1.S4: If there is no Critical Function List or the ISVV objective does not match the criteria used to derive it,
perform a simplified system FMECA along the lines described in Annex E.3.
- MAN.VV.T1.S5: Identify each software product and its supplier. Fill in the error potential questionnaire (see Annex D)
for each software product (by the Error potential assessment described in E.2).
- MAN.VV.T1.S6: Assign ISVV level to each system function based on the software criticality category of the function and
error potential.
Outputs:
- Critical system functions list - Error potential questionnaires - Software criticality scheme - ISVV level definition
4.5.2 Software Technical Specification ISVV level definition
TASK DESCRIPTION
Title: Software Technical Specification ISVV level definition Task ID: MAN.VV.T2
Activity: MAN.VV – ISVV level definition
Start event: PDR – Preliminary Design Review
End event: TAR – Technical Specification Analysis Review
Responsible: This task will be performed by the ISVV customer or the ISVV supplier as determined by the project contract. If carried out by the ISVV supplier, the results shall be reviewed and approved by the ISVV customer.
Objectives:
Inputs:
- From ISVV Customer:
- Technical Specification including Interface Control Documents [from Software Engineering]
- Software safety/dependability analyses based on Technical Specification (if existent) [from Software PA] - Critical system functions list [from System Level ISVV level definition – MAN.VV.T1]
- Error potential questionnaires [from System Level ISVV level definition – MAN.VV.T1] - Software criticality scheme [from System Level ISVV level definition – MAN.VV.T1]
Sub Tasks (Procedure):
- MAN.VV.T2.S1: For each software product implementing critical system functions, identify any SFMECA based on the
Technical Specification available.
- MAN.VV.T2.S2: If an SFMECA exists and the criticality scheme used as a basis is relevant for the ISVV objective, then
it may be used as a basis for deriving the critical software requirements list.
- MAN.VV.T2.S3: If no such analyses have been carried out, the quality is too poor, or the ISVV objective differs from the
presumptions of the SFMECA, perform a simplified SFMECA based on the Technical Specification including Interface Control Documents. Another simplified way of doing this step is described in Annex E.3 (by Simplified SFMECA).
- MAN.VV.T2.S4: Verify the consistency of the SFMECA with the Critical systems function list. If discrepancies are found,
notify the ISVV customer who will have to consider consequences in terms of re-analysis.
- MAN.VV.T2.S5: For each software requirement, derive the software criticality category by identifying the highest
criticality category of any failure mode associated with it.
- MAN.VV.T2.S6: Assign an ISVV level to each software requirement based on the software criticality category of the
requirement and error potential (there is no need to reassess error potential unless different answers to the error potential questionnaire are expected at this level).
Outputs:
- Critical system functions list (update) - Critical software requirements list - Error potential questionnaire - ISVV level definition
4.5.3 Software Design ISVV level definition
TASK DESCRIPTION
Title: Software Design ISVV level definition Task ID: MAN.VV.T3
Activity: MAN.VV – ISVV level definition
Start event: PDR – Preliminary Design Review
End event: TAR – Design Analysis Review
Responsible: This task will be performed by the ISVV supplier and the result reviewed and approved by the ISVV customer.
Objectives:
- Define ISVV level
Inputs:
- From ISVV Customer:
- Technical Specification including Interface Control Documents [from Software Engineering] - Design Definition File: Software architectural design and traceability matrices [from Software
Engineering]
- Design Definition File: Software detailed design and traceability matrices (optional) [from Software Engineering]
- Software safety/dependability analyses based on software architectural design or software detailed design (if existent) [from Software PA].
- From ISVV Supplier:
- Critical system functions list [from System Level ISVV level definition – MAN.VV.T1]
- Error potential questionnaires [from Software Technical Specification ISVV level definition – MAN.VV.T2] - Software criticality scheme [from System Level ISVV level definition – MAN.VV.T1]
- Critical software requirements list [from Software Technical Specification ISVV level definition – MAN.VV.T2]