Quality
Management
Lecture 12 – Software quality
management
doc.dr.sc. Marko Jurčević prof.dr.sc. Roman Malarić
University of Zagreb
Faculty of Electrical Engineering and Computing
The software quality challenge
• Software is (IEEE definition):
• Computer programs, procedures, and possibly associated
documentation and data pertaining to the operation of a computer system.
• The uniqueness of the software development process:
• High complexity, as compared to other industrial products
• Invisibility of the product
• Opportunities to detect defects (“bugs”) are limited to the product development phase
Quality Concepts
• Variation control is the heart of quality control
• Software engineers strive to control the
• process applied
• resources expended
The software quality challenge
• Factors affecting defect detection in software products vs. other industrial products (source: Software Quality Assurance From Theory to Implementation, p. 6)
• significant parts of advanced devices as well as of household machines and other products include embedded software
components (usually termed “firmware”) that are integrated into the product.
• These software components (the firmware) share the same characteristics of the usuall software products
Software quality – IEEE definition
• Software quality is:
• 1. The degree to which a system, component, or process meets specified requirements.
• 2. The degree to which a system, component, or process meets customer or user needs or expectations.
Software quality management and software lifecycle
• Collecting requirements and defining scope of IT project (focus on
verification if defined requirements will be testable)
• Designing the solution (focus on planning test process e.g. what type
of tests will be performed, how they will be performed in context of test environments and test data.
• Solution implementation supported by creating test cases and
scenarios, executing them and registering defects including coordination of fixing them.
• Change management, supported by verification how planned
changes can influence the quality of created solution and eventual change of test plan.
• Closing project, supported by realization number of tests focused on
complex verification of overall quality of created solution. It can include System Integration Tests, User Acceptance Tests and Operational Acceptance Tests.
Causes of software errors
Classification of the causes of software errors:
• (1) Faulty definition of requirements
• faulty definition of requirements, usually prepared by the client, is one of the main causes of software errors
• Erroneous definition of requirements
• Absence of vital requirements
• Incomplete definition of requirements
• Inclusion of unnecessary requirements, functions that are not expected to be needed in the near future
• (2) Client–developer communication failures
• Misunderstandings resulting from defective client–developer
communication are additional causes for the errors that prevail in the early stages of the development process
Causes of software errors ...
Classification of the causes of software errors:
• (3) Deliberate deviations from software requirements
• The developer reuses software modules taken from an earlier project without sufficient analysis of the changes and adaptations needed to correctly fulfill all the new requirements; due to time or budget
pressures, the developer decides to omit part of the required functions in an attempt to cope with these pressures
• (4) Logical design errors
• (5) Coding errors
• (6) Non-compliance with documentation and coding instructions
• (7) Shortcomings of the testing process
• (8) Procedure errors
Software quality assurance – The IEEE definition
• Software quality assurance is:
• 1. A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements.
• 2. A set of activities designed to evaluate the process by which the products are developed or manufactured. Contrast with quality control.
• Software quality assurance is (expanded definition):
• A systematic, planned set of actions necessary to provide
adequate confidence that the software development process or the maintenance process of a software system product conforms to established functional technical requirements as well as with the managerial requirements of keeping the schedule and operating within the budgetary confines.
SQA
• SQA should not be limited to the development process.
• It should be extended to cover the long years of service
subsequent to product delivery. Adding issues directly related to the software product introduces quality issues that integrate
software maintenance functions into the overall conception of SQA.
• SQA actions should not be limited to the technical aspects of the functional requirements, but should include also activities that deal with scheduling and the budget.
The objectives of SQA activities
• Software development (process-oriented):• 1. Assuring an acceptable level of confidence that the software will conform to functional technical requirements.
• 2. Assuring an acceptable level of confidence that the software will conform to managerial scheduling and budgetary requirements.
• 3. Initiating and managing of activities for the improvement and greater efficiency of software development and SQA activities. This means improving the prospects that the functional and managerial requirements will be achieved while reducing the costs of carrying out the software development and SQA activities.
• Software maintenance (product-oriented):
• 1. Assuring with an acceptable level of confidence that the software maintenance activities will conform to the functional technical requirements.
• 2. Assuring with an acceptable level of confidence that the software maintenance activities will conform to managerial scheduling and budgetary requirements.
• 3. Initiating and managing activities to improve and increase the efficiency of software maintenance and SQA activities. This involves improving the prospects of achieving functional and managerial requirements while reducing costs.
Classifications of software requirements
• Classifications of software requirements into software quality factors:
• Product operation factors: Correctness, Reliability, Efficiency,
Integrity, Usability.
• Product revision factors: Maintainability, Flexibility, Testability.
• Product transition factors: Portability, Reusability,
Product operation software quality factors
• Correctness requirements are defined in a list of the software
system’s required outputs
• Reliability requirements deal with failures to provide service. They
determine the maximum allowed software system failure rate, and can refer to the entire system or to one or more of its separate functions.
• Efficiency requirements deal with the hardware resources needed to
perform all the functions of the software system in conformance to all other requirements.
• Integrity requirements deal with the software system security, that is,
requirements to prevent access to unauthorized persons, to
distinguish between the majority of personnel allowed to see the information and a limited group who will be allowed to add and change data.
• Usability requirements deal with the scope of staff resources needed
Product revision software quality factors
• Maintainability requirements determine the efforts that will be
needed by users and maintenance personnel to identify the reasons for software failures, to correct the failures, and to verify the success of the corrections.
• The capabilities and efforts required to support adaptive maintenance activities are covered by the flexibility
requirements. These include the resources (i.e. in man-days) required to adapt a software package to a variety of customers of the same trade, of various extents of activities, of different ranges of products and so on.
• Testability requirements deal with the testing of an information
Product transition software quality factors
• Portability requirements tend to the adaptation of a software
system to other environments consisting of different hardware, different operating systems, and so forth.
• Reusability requirements deal with the use of software
modules originally designed for one project in a new software project currently being developed.
• Interoperability requirements focus on creating interfaces with
other software systems or with other equipment firmware (for example, the firmware of the production machinery and testing equipment interfaces with the production control software).
Interoperability requirements can specify the name(s) of the software or firmware for which interface is required. They can also specify the output structure accepted as standard in a specific industry or applications area.
The SQA system – an SQA architecture
• SQA system always combines a wide range of SQA components, all of
which are employed to challenge the multitude of sources of software errors and to achieve an acceptable level of software quality
• SQA system components can be classified into six classes:
• Pre-project components. To assure that (a) the project commitments have been adequately defined considering the resources required, the schedule and budget; and (b) the development and quality plans have been correctly determined.
• Components of project life cycle activities assessment. The project life cycle is composed of two stages: the development life cycle stage and the operation– maintenance stage.
• The development life cycle stage components detect design and programming errors. Its components are divided into the following sub-classes: Reviews, Expert opinions,
Software testing.
• Components of infrastructure error prevention and improvement
• eliminate or at least reduce the rate of errors, based on the organization’s accumulated SQA experience
• Components of software quality management.
• control of development and maintenance activities and the introduction of early
managerial support actions that mainly prevent or minimize schedule and budget failures and their outcomes
• Components of standardization, certification, and SQA system assessment • Organizing for SQA – the human components
Software project life cycle components
• main components are:
• Reviews
• Expert opinions
• Software testing
• Software maintenance
• Assurance of the quality of the subcontractors’ work and the customer supplied parts.
Reviews
• The design phase of the development process produces a variety of documents.
• The printed products include design reports, software test
documents, software installation plans and software manuals, among others. Reviews can be categorized as formal design reviews (DRs) and peer reviews.
Software Reviews
• Purpose is to find defects (errors) before they are passed on to another software engineering activity or released to the
customer
• Software engineers (and others) conduct formal technical reviews (FTR) for software engineers
• Using formal technical reviews (walkthroughs or inspections) is an effective means for improving software quality
Review Roles
• Presenter (designer/producer).
• Coordinator (not person who hires/fires).
• Recorder
• records events of meeting
• builds paper trail
• Reviewers
• maintenance oracle
• standards bearer
• user representative
Formal Technical Reviews
• Involves 3 to 5 people
• Advance preparation (no more than 2 hours per person) required
• Duration of review meeting should be less than 2 hours
• Focus of review is on a discrete work product
• Review leader organizes the review meeting at the producer's request
• Reviewers ask questions that enable the producer to discover his or her own error (the product is under review not the producer)
• Producer of the work product walks the reviewers through the product
• Recorder writes down any significant issues raised during the review
• Reviewers decide to accept or reject the work product and whether to require additional reviews of product or not
Why do peer reviews
• To improve quality
• Catches 80% of all errors if done properly; both coding errors and design errors
• Enforce the spirit of any organization standards
Expert opinions
• Expert opinions support quality assessment efforts by introducing additional external capabilities into the organization’s in-house development process.
Statistical Quality Assurance
• Information about software defects is collected and categorized
• Each defect is traced back to its cause
• Using the Pareto principle (80% of the defects can be traced to 20% of the causes) isolate the "vital few" defect causes
Software testing
• Software tests are formal SQA components that are targeted
toward review of the actual running of the software.
• tests are based on a prepared list of test cases that represent a variety of expected scenarios
• examine software modules, software integration, or entire software packages (systems)
• Software testing programs are constructed from a variety of
tests, some manual and some automated. All tests have to be designed, planned and approved according to development procedures.
• test report will include a detailed list of the faults detected and
recommendations about the performance of partial or complete recurrent tests following a subsequent round of corrections
Software maintenance components
• Software maintenance services vary in range and are provided for extensive periods, often several years.
• Corrective maintenance – User’s support services and
correction of software code and documentation failures.
• Adaptive maintenance – Adaptation of current software to new
circumstances and customers without changing the basic software product.
• Functionality improvement maintenance – The functional and
performance - related improvement of existing software, carried out with respect to limited issues
Infrastructure components for error prevention and improvement
• goals of SQA infrastructure are the prevention of software faults or, at least, the lowering of software fault rates, together with the improvement of productivity
• This class of SQA components includes:
• Procedures and work instructions
• detailed definitions for the performance of specific types of
development activities in a way that assures effective achievement of quality results
• Templates and checklists
• Staff training, retraining, and certification
• Preventive and corrective actions
• Configuration management
Verification and Validation
• Verification and Validation are independent procedures that are used together for checking that a product, service, or system meets requirements and specifications and that it fulfills its intended purpose
• The PMBOK guide :
• "Validation - The assurance that a product, service, or system
meets the needs of the customer and other identified stakeholders. It often involves acceptance and suitability with external
customers. Contrast with verification.„
• "Verification. The evaluation of whether or not a product, service, or system complies with a regulation, requirement, specification, or imposed condition. It is often an internal process. Contrast with validation."
Validation Perspectives
• Reliability validation
• Does measured system reliability meet its specification?
• Is system reliability good enough to satisfy users?
• Safety validation
• Does system operate so that accidents do not occur?
• Are accident consequences minimized?
• Security validation
Validation Techniques
• Static techniques
• design reviews and program inspections
• mathematical arguments and proof
• Dynamic techniques
• statistical testing
• scenario-based testing
• run-time checking
• Process validation
• SE processes should minimize the chances of introducing system defects
Static Validation Techniques
• Concerned with analysis of documentation
• Focus is on finding system errors and identifying potential problems that may arise during system operation
• Documents may be prepared to support static validation
• structured arguments
Poka-Yoke Devices
• mechanisms that lead to the prevention of a potential quality/function problem before it occurs or to the quick detection of a quality/function problem if one is introduced
• a simple, cheap, part of the engineering process, and are
located near the process task where the mistakes are likely to occur
ISO/IEC 25010:2011
• ISO/IEC 9126 Software engineering — Product quality was
an international standard for the evaluation of software quality.
• It has been replaced by ISO/IEC 25010:2011
• The standard is divided into four parts:
• quality model
• external metrics
• internal metrics
ISO/IEC 25010:2011
• Quality Model - classifies software quality in a structured set of characteristics and sub-characteristics
• Functionality - A set of attributes that bear on the existence of a set of functions and their specified properties
• Reliability - A set of attributes that bear on the capability of software to maintain its level of performance under stated conditions for a stated period of time
• Usability - A set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by a stated or implied set of users
• Efficiency - A set of attributes that bear on the relationship between the level of performance of the software and the amount of resources used, under stated conditions
• Maintainability - A set of attributes that bear on the effort needed to make specified modifications
• Portability - A set of attributes that bear on the ability of software to be transferred from one environment to another
References
• Software Assurance Definitions,
http://www.hq.nasa.gov/office/codeq/software/umbrella_defs.htm
• Daniel Galin, Software Quality Assurance - From theory to
implementation, Pearson – Addison Wesley, 2004
• Software quality management,
• http://en.wikipedia.org/wiki/Software_quality_management
• http://en.wikipedia.org/wiki/Software_quality
• Verification and validation,
