Software Quality Engineering
BS(SE)-VI
Dr. Assad Abbas
Department of Computer Science
What is Software Quality?
n
No commonly agreed definition
n
IEEE definition
“The degree to which a system, component, or process
meets specified requirements (Philip Crosby)”
5
Emphasis here is on specifications
“The degree to which a system, component, or process
meets customer or user needs or expectations (Joseph
M. Juran)”
What is Software Quality?
n
Pressman believes that Software quality is :
“Conformance to explicitly stated functional and
performance requirements, explicitly documented
People’s Quality Expectations
n
In general, people’s quality expectations for software
systems they use and rely upon are two-fold:
5
The software systems must do what they are
supposed to do. In other words, they must
do
the
right things.
g
For example, an airline reservation system is supposed
to handle reservations, not intended to fly airplanes
automatically
5
They must perform these specific tasks correctly or
satisfactorily. In other words, they must
do
the things
right.
g
the system should help travel agents or individual
Reasons Software Quality is Important
n
Safety
n
Cost
n
Customer satisfaction
Some Historical Facts
n
Software bugs caused:
5
A massive blackout cut off electricity to 50 million
people in eight US states and Canada
5
Technical problems with the baggage system at
Heathrow’s terminal 5 caused thousands of the
passengers to wait for their baggage
n
A study commissioned by the National Institute of
Quality Control vs. Quality Assurance
n
Quality Control (QC) is a set of activities for ensuring quality
in products. The activities focus on
identifying defects
in the
actual products produced.
5 Quality control, therefore, is a reactive process.
5 The goal of QC is to identify defects after a product is developed and before it's released
n
Quality Assurance is a set of activities for ensuring quality
in
the processes
by which products are developed
5 QA aims to prevent defects with a focus on the process used to make the product. It is a proactive quality process
5 The goal of QA is to improve development and test processes so that defects do not arise when the product is being
Testing, Quality Assurance, and Software Quality
Engineering
n By running the software system or executing its
prescribed functions, testers can determine if the observed system behavior conforms to its
specifications or requirements
n Beyond testing, there are many other QA
alternatives supported by related techniques and activities, such as inspection, formal verification, defect prevention, and fault tolerance
n In addition, all these QA activities need to be
managed in an engineering process we call the software quality engineering process, with quality goals set early in the product development, and strategies for QA selected, carried out, and
Why Software Quality Engineering?
n
Why software?
5
Because in contemporary social life software, systems
and services rendered by software are omnipresent,
beginning with the watches we wear, ending with nuclear
electricity plants or spaceships.
n
Why quality?
5
Because if these instances of software work without the
required quality we may be late, dead, or lost in space.
n
Why engineering?
Importance w.r.t. process
n
Specifications
5 The correctness, completeness, and consistency of the
requirements model will have a strong influence on the quality of all work products that follow
n
Design
5 Every element of the design model should be assessed by the software team to ensure that it exhibits high quality and that the design itself conforms to requirements
n
Construction
5 Source code and related work products (e.g., other descriptive information) must conform to local coding standards and exhibit characteristics that will facilitate maintainability
n
Conformance
The environments for which SQA methods are
developed
n
Important to note that variety of professionals, such
as students, amateur developers, professionals in
engineering, economics, management and other
fields, software teams, departments of large and
small industrial applications and all those who
n
Contractual conditions
n
Subjection to customer-supplier relationship
n
Requirement for teamwork
n
Need for cooperation and coordination with other
development teams
n
Need for interfaces with other software systems
n
Need to continue carrying out a project while the
team changes
n
Need to continue maintaining the software system
for years
The characteristics of the SQA environment process
n
Contractual Conditions
5 As a result of the commitments and conditions defined in the contract between the software developer and the customer, the activities of software development and maintenance need to cope with:
g A defined list of functional requirements that the developed
software and its maintenance need to fulfill.
g The project budget. g The project timetable
n
Subjection to customer-supplier relationship
5 Continuous cooperation with the customer
g Request for changes, criticism/feedback, approval of changes
The characteristics of the SQA environment process
n
Requirement for teamwork
5 The following factors usually motivate the establishment of a project team rather than assigning the project to one
professional
g Timetable (schedule) requirements
g The need for a variety of specializations to carry out the project. g The desire to benefit from professionals’ mutual support and
review for the enhancement of project quality.
n
Cooperation and coordination with other development teams
5 Cooperation may be required with:
g Other software development teams in the same organization. g Hardware development teams in the same organization.
g Software and hardware development teams of other suppliers. g Customer software and hardware development teams that take
The characteristics of the SQA environment process
n
Interfaces with other software systems
5
One can identify the following main types of interfaces:
g Input interfaces, where other software systems transmit data to
your software system.
g Output interfaces, where your software system transmits
processed data to other software systems.
g Input and output interfaces to the machine’s control board, as in
medical and laboratory control systems, metal processing equipment, etc.
n
Need to continue carrying out a project while the team
changes
The salary software system – an example of software
interfaces
Salary processing
system Attendance
control system
Bank information
systems
Money transfers to employees’ bank account accounts
Monthly attendance report, including overtime calculations
Input interface
The characteristics of the SQA environment process
n
Need to continue maintaining the software system
for years
5
Customers who develop or purchase a software
system expect to continue utilizing it for a long period,
usually for 5–10 years. During the service period, the
need for maintenance will eventually arise. In most
cases, the developer is required to supply these
services directly.
5
Internal “customers”, in cases where the software has
been developed in-house, share the same
ISO 9001:2000 Standard
n
ISO 9001:2000 is the quality assurance standard that applies
to software engineering.
n
The standard contains 20 requirements that must be present
for an effective quality assurance system.
n
The requirements defined by ISO 9001:2000 address topics
such as:
5 management responsibility, quality system, contract review, design control, document and data control, product identification and
Quality Perspectives and Expectations
n
In Kitchenham & Pfleeger (1996):
5
Transcendental view: seen/not-defined
5
User view: fitness for purpose
5
Manufacturing view: conformance to standards
5
Product view: inherent characteristics
People’s Roles and Responsibilities
n About software quality, people may have different views and expectations based
on their roles and responsibilities
n Mainly two groups:
5 Consumers of software products and services
g Customers, users, non-human users (other systems) g External view (black box)
g Quality expectations:
u Can be different for different users ( usability for novice and reliability for sophisticated users)
u Correctness, performance, maintainability
5 Producers of software products and services
g anyone involved with the development, management, maintenance, marketing,
and service of software products
g Internal view (white box) g Quality expectations:
u Fulfil contractual obligations
u Design, size, complexity, change
Quality Frameworks and ISO-9126
n ISO-9126—hierarchical framework for quality definition, organized
into quality characteristics and sub-characteristics with six top-level
quality characteristics/dimensions:
5 Functionality: what is needed?
g Suitability, Accuracy, Interoperability, Security
5 Reliability: function correctly
g Maturity, fault tolerance, recoverability
5 Usability: effort to use
g Understandability, learnability, operability
5 Efficiency: resource needed
g Time, resource
5 Maintainability: correct/improve/adapt
g Analyzability, changeability, stability, testability
5 Portability: one environment to another
Alternative frameworks and focus on correctness
n
Quality measure for IBM,s software products
5 CUPRIMDS (capability, usability, performance, reliability, installation, maintenance, documentation, and service)
n Quality attributes for Web-based applications
5 reliability, usability, and security—primary attributes
5 availability, scalability, maintainability, and time to market—secondary attributes
g performance (or efficiency) and reliability would take precedence over usability and
maintainability for real-time software products. On the contrary, it might be the other way round for mass market products for end users.
n
Correctness is typically the most important aspect of quality
CORRECTNESS AND DEFECTS
n
Error:
5
incorrect/missing human action – conceptual
mistakes
nFault:
5
An incorrect step, process, or data definition in a
computer program.
n
Failure:
5
The inability of a system or component to perform its
required functions within specified performance
Correctness and Defects
n
We also extend errors to include
error sources or the root
causes
for the missing or incorrect actions, such as human
misconceptions, misunderstandings, etc.
n
Failures, faults, and errors are collectively referred to as
defects in literature.
n
Software problems or defects, are also commonly referred to
as “bugs”. However, the term bug is never precisely defined
n
Moreover, we use
defect detection and removal
for the overall
concept and activities related to what many people commonly
call
“debugging”.
5 Specific activities related to defect discovery, including testing, inspection, etc.
Concepts Related to Defects
n The relationship is not necessarily 1-to- 1:
n A single error may cause many faults, such as in the case that a
wrong algorithm is applied in multiple modules and causes multiple faults, and a single fault may cause many failures in repeated executions.
n Conversely, the same failure may be caused by several faults,
such as an interface or interaction failure involving multiple
modules, and the same fault may be there due to different errors.
n Sometimes, an error source, such as e5, may not cause any fault
injection, and a fault, such as f4, may not cause any failure, under the given scenarios or circumstances. Such faults are typically
Concepts Related to Defects
n
It should be noted that only a portion of the software faults,
and in some cases only a small portion of them, will turn into
software failures in either the early or later stages of the
software’s application.
n
Other software faults will remain hidden, invisible to the
Correctness Oriented Properties and Measurements
n
Failure properties and direct failure measurement
5 Failure properties include information about the specific failures, what they are, how they occur, etc. These properties can be
measured directly by examining failure count, distribution, density etc.
n
Failure likelihood and reliability measurement
5 How often or how likely a failure is going to occur is of critical concern to software users and customers. This likelihood is
Correctness Oriented Properties and Measurements
n
Failure severity measurement and safety assurance
5 Accidents, which are defined to be failures with severe
Defects in the context of QA and quality engineering
n
Three generic ways to deal with defects include:
5
defect prevention
5
defect detection and removal
5
defect containment
n
Quality engineering can also be viewed as
defect
management
. In addition to the execution of the
planned QA activities, quality engineering also
includes:
5
quality planning before specific QA activities are
carried out
5
measurement, analysis, and feedback to monitor and
Causes of Software Errors
n Faulty definition of requirements
5 Erroneous definition of requirements
5 Incomplete definition of requirements
5 Inclusion of unnecessary requirements, functions that are not expected to be needed in the near future
n Client-developer communication failures
5 Misunderstanding of (i) client’s instructions as stated in the requirement documents, (ii) requirement changes, (iii) client’s response to design problem
n Deliberate deviation from requirements
n Logical design errors
5 Process definitions that contain sequencing errors
5 Erroneous definition of boundary conditions
5 Omission of required software states
Causes of Software Errors
n Coding errors
5 misunderstanding the design documentation, linguistic errors in the programming languages, errors in the application of CASE and other development tools, errors in data selection, and so forth.
n Non-compliance with documentation and coding instructions
5 Team members who need to coordinate their own codes with code modules developed by “non-complying” team members can be expected to encounter more than the usual number of difficulties when trying to understand the
software developed by the other team members.
5 Individuals replacing the “non-complying” team member (who has retired or been promoted) will find it difficult to fully understand his or her work.
5 The design review team will find it more difficult to review a design prepared by a non-complying team.
n Shortcomings of the testing process
5 Incomplete test plans; failure to report, document and correct detected errors;
n Procedure errors
n Documentation errors
