• No results found

Applying Quality Assurance in Software Acquisition and Development

N/A
N/A
Protected

Academic year: 2020

Share "Applying Quality Assurance in Software Acquisition and Development"

Copied!
10
0
0

Loading.... (view fulltext now)

Full text

(1)

 

APPLYING QUALITY ASSURANCE IN SOFTWARE ACQUISITION

AND DEVELOPMENT

Subir K. Sen1,

1

D.Sc. P.E., MASCE, United States Department of Energy

ABSTRACT

With the explosive use of software in all facets of technological development, software quality assurance (SQA) has increasingly become a must needed tool to assure that the software will perform its intended function and not cause any adverse impact by its failure to perform. This is particularly important for a nuclear facility where software failure can have potentially adverse impact on the safety and health of the public, workers or the environment. It is therefore necessary to control the use of software in safety applications through established procedures. This paper examines how SQA is applied in the acquisition and development of software within the United States Department of Energy (DOE).

Starting with a general discussion of the basic quality assurance (QA) principals and controls within the framework of a quality assurance program (QAP), the paper reviews the regulations, standards, and guidance that have been developed in DOE that need to be applied to assure software quality. The SQA requirements are reviewed based on software types (acquired, custom-developed or configurable) and software categories (nuclear safety, other safety-affecting, mission critical or general software) using a risk based graded approach. The safety software categorization is reviewed with respect to its function in credited hazard controls as described in the documented safety analysis or hazard analysis. The QA aspects of software life-cycle work activities that are generally followed in the industry for software development (e.g., quality planning, requirements identification, configuration management, design, verification and validation etc.), are discussed. A systematic approach to software management and guidance on selection and implementation of the SQA work activities commensurate with the software complexity and consequence of failure are presented. Special considerations for use in nuclear safety applications of commercially available software that has not been developed under an approved QAP are discussed. Finally, an overview of the process and procedures used in DOE to assess software quality and maintain a registry of SQA evaluated software is reviewed.

 

INTRODUCTION

With the increasing use of software in all facets of technological development, software quality assurance (SQA) has increasingly become a must needed requirement to assure that the software will perform its intended function and not cause any adverse impact by its failure to perform. This is

particularly important for a nuclear facility where software is extensively used in the design and operation of the facility and software failure can have potentially adverse impact on the safety and health of the public and the workers and the environment. It is therefore necessary to control the use of software in safety applications through established procedures. This paper examines the SQA requirements for acquired and developed software within the United States Department of Energy (DOE).

SOFTWARE QUALITY ASSURANCE REQUIREMENTS

The DOE’s ten QA criteria for management, performance and assessment originate from Title 10 of the United States Code of Federal Regulations (CFR), 10 CFR Part 830, Subpart A, Quality Assurance

(hereafter the QA Rule). The QA Rule establishes the “quality assurance requirements for contractors conducting activities, including providing items or services that affect, or may affect, nuclear safety of DOE nuclear facilities.” DOE Order (O) 414.1D, Quality Assurance (hereafter the QA Order)

(2)

 

use in the definition of work and requires that all software meet the applicable QA criteria and prescribes special requirements for safety software.

Software Quality Assurance Program

The QA Order requires that all software related activities shall be performed under a DOE approved Quality Assurance Program (QAP) using a graded approach and sound software engineering practices. The QAP should identify the applicable industry and consensus standards used to perform the SQA work activities.

The QA Rule and the QA Order define “graded approach” as the process of ensuring that the levels of analyses, documentation, and actions used to comply with requirements is commensurate with:

 The relative importance to safety, safeguards and security;  The magnitude of any hazard involved;

 The life-cycle stage of a facility or item;  The programmatic mission of a facility;

 The particular characteristic of a facility or item;

 The relative importance to radiological and non-radiological hazards; and  Any other relevant factors.

When applying the graded approach to SQA, the following should be considered:

 The software’s intended function and its relative importance to mission, safety and facility operations;

 The relative consequences of software failure;  The complexity of the software; and

The software type and category.

Software Classification

Software is classified by type and category and is assigned an appropriate grading level to establish the requirements to be applied using the graded approach.

Software Types

Software applications are classified as acquired, configurable, or custom-developed software types. Software belonging to one of these software types can have features and characteristics that can change the designation to a different software type.

Acquired Software

Acquired software is generally supplied through basic procurements, two-party agreements, or other contractual arrangements. Acquired software includes commercial off-the-shelf (COTS) software that will be used as provided. Acquired software also includes government-furnished software, software that is downloaded at no cost to the user, and software acquired from a parent corporate entity or from an open source distributor. Acquired software can be used for a broad range of activities, such as: design analysis, safety and hazard analysis, modeling of physical phenomena and engineering.

Configurable Software  

(3)

 

important to differentiate between QA of the underlying (e.g., vender supplied) software or firmware and QA of the configurable software itself.

Custom-Developed Software  

Custom-developed software is software developed for a specific purpose. Custom software may be developed by DOE, by an existing DOE contractor, or by a qualified software development company selected via the procurement process. Examples of custom-developed software include custom material inventory and tracking software, custom data management software, custom accident consequence analysis software, custom control system software, custom operations software, and custom research software. Custom-developed software also includes calculation applications such as spreadsheet

programs (e.g., Excel™ visual basic applications) or programs developed using such tools as Mathcad,™ MatLab,™ etc.

Software Categories 

There are two software categories: (1) safety software and (2) other software.  

Safety Software

Safety software is defined by the QA Order and includes:  Safety System Software (SSS);

 Safety and Hazard Analysis Software and Design Software (SHADS); and  Safety Management and Administrative Controls Software (SMACS).

Safety System Software (SSS)

The QA Order defines SSS as: Software for a nuclear facility that performs a safety function as part of structures, systems and components (SSC) and is cited in either (a) a DOE-approved documented safety analysis; or, (b) an approved hazard analysis. The use of the term “nuclear facility” includes both reactor and nonreactor nuclear facilities.

Safety and Hazard Analysis Software and Design Software (SHADS)

The QA Order defines SHADS as software that is used to classify, design, or analyze nuclear facilities. This software is not part of an SSC but helps to ensure proper accident or hazards analysis of nuclear facilities and selection of hazard controls and physical features to prevent or mitigate a hazard to the workers, the public and the environment. It also includes software used in the design of a SSC that performs a safety function.

Safety Management and Administrative Controls Software (SMACS)

SMACS is defined as a software that performs a hazard control function in a nuclear facility or in support of radiological safety management programs, technical safety requirements or other software that performs a control function necessary to provide adequate protection from nuclear facility or radiological hazards. This software supports eliminating, limiting, or mitigating nuclear hazards to workers, the public, or the environment. This software does not directly affect a safety class (SC) or safety significant (SS) SSC safety function. Safety management programs and administrative controls relied upon for safety are identified in the facility documented safety analysis (DSA) and technical safety requirements (TSRs).

Other Software (OSW)

(4)

 

applicable QA criteria and requirements specified in the QA Order, using a graded approach. The following describes the different categories of OSW.

 Safety-Affecting Software;  Critical Software; and  General Software.

Safety-Affecting Software (SAS)

SAS includes software that performs a hazard control function that does not meet the nuclear safety software definitions in the QA Order. This software contributes to the control of radiological, biological, chemical, and physical hazards defined by the site’s Integrated Safety Management System hazard analysis process. Such software may be relied upon for the control of hazardous chemicals, explosives, asphyxiants, high energy chemical reactions, laser or particle beams, combustible gases, toxic materials, nerve agents or biological agents. SAS includes software whose failure could have a safety or health consequence.

Critical Software (CS)

CS includes software needed for complying with requirements for operation of a facility, process or project. Examples include software for environmental protection, emergency preparedness and response, communication, accident mitigation and recovery, regulatory compliance, hazardous inventory management, waste management, safeguards and security, machine control or machine protection systems and mission critical and some research and development activities. The failure of CS does not have a direct health or safety consequence.

General Software (GS)

GS includes software not falling into any of the software subcategories described above. Failure of this software would be of low consequence.

Safety Software Determination

The determination of what constitutes safety software should be made based on its application and the safety consequences of its postulated failure to perform its required function. The process for identifying safety software is tied to the formal safety analysis and hazard assessment process for the nuclear facility. Since software can have both safety and non-safety applications, a determination should be made whether the non-safety application could adversely affect the safety application. This

determination is also important where software is used to monitor radiological inventory as a means of categorizing a nuclear facility.

Software Grading Levels

The QA Order requires sites to develop and implement safety software grading levels approved by DOE. Software must be appropriately categorized as safety software prior to addressing safety software grading levels. Guidance on how SQA work activities can be graded is discussed in DOE (2016) draft. Grading can be applied to an entire system or to individual software components.

(5)

 

Table 1: Suggested Safety Software Grading Levels

Grading Level Description Grading Levels

Safety software must execute correctly or the intended use of the system/software will not be realized, causing one or more of the following grave consequences:

 Adversely affect the safety function of a SC or SS SSC credited in a DOE-approved DSA;

 Compromise a limiting condition for operation or limiting control settings of a SC or SS system, as stated in a TSR;

 Adversely affect the safety or hazard control function of other systems, such as toxic or chemical protection systems, credited in either (a) a DOE-approved DSA or (b) an approved hazard analysis;

 Result in misclassification of facilities or non-conservative hazard analysis, accident analysis or hazard control selection and misclassification of SSC per 10 CFR Part 830 or DOE-STD-3009-2014;or

 Adversely affect the safety function of a specific administrative control (SAC), or the supporting SSCs to implement a SAC, credited in a DOE-approved DSA.

A

Safety software must execute correctly or the intended use of the system/software will not be realized, causing one or more of the following serious consequences:

 Adversely affect the hazard control function of defense-in-depth SSCs identified in a safety management program in a DOE-approved DSA;

 Result in inaccurate or non-conservative design and analysis of SC or SS SSCs credited in a DOE-approved DSA; or

 Result in an incorrect analysis, monitoring, alarming, or recording of hazardous exposures to workers or the public.

B

Safety software must execute correctly or an intended hazard control function will not be realized, causing minor consequences. This grading level will apply to all safety software not graded as level A or B.

(6)

 

Table 2 Suggested OSW Grading Levels

Grading Level Description Risk Grading Levels

OSW must execute correctly or the intended use of the system/software will not be realized, causing one or more of the following consequences:

 Loss of human life or injuries resulting in long term disability;

 Significant offsite hazardous (non-radiological) release to the environment; or

 Critical impact on mission or performance objectives

High to Very High

1

OSW must execute correctly or the intended use of the system/software will not be realized, causing one or more of the following consequences:

 Injuries resulting in short term disability;

 Incorrect analysis, monitoring, alarming, or recording of hazardous exposures to workers or the public from non-nuclear facilities;

 Significant onsite hazardous (non-radiological) release to the environment;

 Significant regulatory violation; or

 Serious impact on mission or performance objectives.

Moderate to High

2

OSW must execute correctly or the intended use of the system/software will not be realized, causing one or more of the following consequences:

 Non-reportable injury;

 Release within a site building or facility; or

 Moderate impact on mission or performance objectives.

Low to Moderate

3

OSW must execute correctly or the intended use of the system/software will not be realized, causing minor consequences. This grading level will apply to all OSW not graded as level 1, 2 or 3.

Low 4

SOFTWARE LIFE-CYCLE ACTIVITIES

Overview of Software Related Work Activities

It is widely accepted that the assessment of software cannot be limited to verification and testing of the end product (i.e., the computer code). Other factors such as the quality of the development processes, adequacy of the product and the competence and qualifications of the staff involved in all of the life-cycle phases have important impact on the quality of the software implementation.

The QA Order prescribes the following SQA work activities for safety software that are to be implemented as applicable using selected consensus standards and established grading levels approved by DOE.

1. Software project management and quality planning; 2. Software risk management;

3. Software configuration management; 4. Procurement and supplier management;

5. Software requirements identification and management; 6. Software design and implementation;

7. Software safety analysis and safety design methods; 8. Software verification and validation;

(7)

 

10. Training of personnel in the design, development, use, and evaluation of safety software.

Although the above SQA work activities are specifically intended to be applicable for safety software, they also represent standard software life-cycle development activities generally followed in the software industry. These work activities may therefore be used for OSW. Other approaches may also be acceptable as long as they meet the DOE QA requirements. Additional information including guidance specific to each of the ten SQA work activities is provided in the DOE (2005) guidance document.

Implementation of Software Related Work Activities

The six-step process illustrated in Figure 1 below may be used for selecting and implementing SQA work activities.

Figure 1 Six-Step Approach to Selecting and Implementing Work Activities

The first step is to identify consensus standards, as required by the QA Order and in the DOE approved QAP.

The second step is to determine the software category (i.e., safety software or OSW), subcategory and grading level.

The third step is to determine the applicability of the work activities consistent with the software type. Identify and apply only those work activities that are: (a) within the scope of the software project and (b) within the control of the project. For example, in the case of acquired software, the design and implementation work activity is not applicable for the project.

The fourth step is to define the work activities with the appropriate elements from the selected consensus standards. This is done by specifying which elements, in whole or in part, of the selected consensus standards describes the work activity. When standards do not fully address the requirements, gaps should be addressed in the QAP.

The fifth step is to define appropriate requirements to be used for each work activity based on the software category and grading level.

The final step is to grade by applying appropriate rigor.

Detailed Guidance on Software Related Work Activities

The SQA work activities which include tasks that are specific to software life-cycle phases, provide the basis for planning, implementing, maintaining, and operating software. Note that:

 These work activities describe the life-cycle process generally employed in software development;

 Not all work activities are applicable for all types of software;

(8)

 

personnel and implemented without any adverse impact on the software’s ability to perform its intended function;

 Greater rigor is applied throughout where safety is involved; and

 These work activities are configured to address all ten QA criteria in the QA Order.

SQA Work Activity Documentation

The SQA work activities with the corresponding documentation are listed in Table 3. This documentation may be standalone or may be combined with other documents depending upon the type of software and scope of its application.

Table 3: Software Quality Assurance Work Activities and Related Documentation

SQA Work Activity Potential SQA Documents

1. Software Project Management and Quality Planning

- Software Project Management Plan and/or

- Software Quality Assurance Plan

- Software Safety Plan (optional)

2. Software Risk Management - Various document types can be used to cover risk management

3. Software Configuration Management - Software Configuration Management Plan or related documents

4. Procurement and Supplier Management - Contractual documents or other procurement and use agreement documentation

- Software concept of operation (optional) 5. Software Requirements Identification and

Management

- Software requirements specification or related document

6. Software Design and Implementation - Software design description, Model Description, Programmer’s Manual, Theory Manual, User Manual or other related documents

7. Software Safety Analysis and Safety Design Methods

- Software design description

- Software Safety Analysis documentation 8. Software Verification and Validation - Verification and Validation Report

- Test Plan, Test Case Description and Outcome Report; and other testing documents

- Test results and evidence that code output was compared to experimental results or against equivalent output from an independent code or by another accepted approach and differences resolved 9. Problem Reporting and Corrective Action - Software Error Notification and Corrective Action

Report 10. Training of Personnel in the Design,

Development, Use and Evaluation of Safety Software

- User Instructions or User Manuals, Installation Manual

(9)

 

COMMERCIAL GRADE DEDICATION OF COMPUTER PROGRAMS AND SOFTWARE SERVICES

Commercial grade dedication (CGD) consists of qualifying safety-related items for use, that were not designed, developed, or manufactured or services that were not provided in accordance with an acceptable QAP. CGD is part of the acquisition process and is intended to provide reasonable assurance that the computer program or service will perform its intended safety function. Commercial grade software services may include installation of computer programs, operating system updates and computer program patches, development or modification of computer programs that perform a safety function, performance of independent verification and validation activities, or other technical support activities.

For computer programs or software services, CGD involves verifying that a commercial grade computer program or software service meets functional requirements, critical characteristics, and established acceptance criteria. Detailed requirements of the CGD process are found in of American Society of Mechanical Engineers (ASME) NQA-1, (2009). Additional guidance is available in Electric Power Research Institute (EPRI) reports, and DOE’s Office of Environmental Management,(2011).

MANAGEMENT OF CENTRAL REGISTRY AND TOOLBOX CODES

A toolbox code is a term used to identify software qualified to be listed in the DOE Safety Software Central Registry (CR) ( http://energy.gov/ehss/safety-software-quality-assurance-central-registry) that is used primarily for DOE safety analyses. These codes have use throughout the DOE complex and are of appropriate qualification for use at DOE sites. The listed versions of these codes have been evaluated using SQA criteria for safety software and have been found safe to use consistent with the guidance document prepared for the specified version of each code. However, as these toolbox code versions are being updated to newer versions, they are evaluated to current DOE SQA requirements.

For each code evaluation, a team is assembled with team members having in depth knowledge of the code and the DOE SQA requirements. The team’s evaluation and recommendations for correcting any SQA deficiencies are passed on to the code developer for corrective action. Following acceptable implementation of the corrective action, the code version is approved as a toolbox code and listed in the DOE CR. Analysts using codes listed in the CR should not have to further justify their SQA pedigree. However, other SQA requirements according to the established site procedures should be followed.

Most of the toolbox codes currently in the CR were developed under the sponsorship of federal agencies other than DOE. Access to the toolbox codes is subject to agreements, conditions, and restrictions established by the code custodians or federal agencies. Use of the CR toolbox codes is not mandatory. Use of codes other than toolbox codes or use of other toolbox code versions not listed in the CR should be justified by showing that applicable DOE SQA requirements for the application are met.

CONCLUSIONS

DOE has developed the QA Rule, QA Order and Guidance document toprovide requirements and acceptable methods for implementing appropriate SQA for various types and categories of software using a graded approach. DOE has also established the Central Registry of approved toolbox codes that meet DOE SQA requirements.

REFERENCES

DOE 10 CFR Part 830, (2001) Nuclear Safety Management, January 10, 2001

DOE O 414.1D, Change 1, (2013) Quality Assurance, May 8, 2013

DOE G 414.1-4, (2005) Safety Software Guide for Use with 10 CFR Part 830, Subpart A, Quality

(10)

 

DOE G 414.1-4A, (2016) Software Quality Assurance Guide for Use with DOE O 414.1D Draft

ASME NQA-1-2008 with the NQA-1a-2009 (2009) Addenda, Quality Assurance Requirements for

Computer Software for Nuclear Facility Applications.

Figure

Table 1:  Suggested Safety Software Grading Levels
Figure 1 Six-Step Approach to Selecting and Implementing Work Activities
Table 3:  Software Quality Assurance Work Activities and Related Documentation

References

Related documents

In Article 74(1) of our 1950 constitution had provided that “there shall be a Council of Ministers with the Prime Minister to aid and advice the president in the exercise of

A similar analysis was presented in Hutchinson, Wright, and Mi- lan ( 2011 ) for Solar Cycle 23 but concluded that storm main phase duration increases with intensity to a point,

that you are using is successful. If it’s not then, actually you should go back and to find some other way to do some lesson, but it’s also to find out what the kids

We assessed patterns of neutral genetic variation across this species’ range using a variety of metrics to determine whether genetic diversity and population structure were

αρ16g τoυE;,.. ,oλα αυτd πoυ μ6λιζ αγ6φε- ρα, απoτελoυν ιαι τoυζ λoγoυζ, γtα τol.lζ oπoloυg μπoρotμε να εLπiξoυμε 6τι oι Λα- xεδαιμ6νιoι Θα

Table 4.12: Top three used SM categories for PMBOK process groups Table 4.13: Least frequently used SM category by PMBOK knowledge area Table 5.1: Participants distribution

The preliminary AA-QS values derived in this publication for both the fresh water and Baltic sea were compared for those five active substances and one metabolite that have

In summary, several key differences were observed in spending patterns by SNAP status, in particular with respect to sugar-sweetened beverages, red meat and cold convenience foods