Software Engineering: Analysis and Design - CSE3308

(1)

CSE3308 - Software Engineering: Analysis and Design, 2004 Lecture 10A. 1

Software Engineering: Analysis and

Design - CSE3308

Software Quality

CSE3308/DMS/2004/25

Monash University - School of Computer Science and Software Engineering

Lecture Outline

uWhat is Software Quality?

uHow can we measure Software Quality?

uSoftware Quality Activities

vSoftware Quality Assurance vSoftware Quality Planning vSoftware Quality Control uQuality Standards

vAS3563 and ISO9000

uBeyond Quality Standards

vCapability Maturity Model vPersonal Software Process

(2)

What is Software Quality?

“The problem of quality management is not what people don’t know about it. The problem is what they think they do know….

Quality has much in common with sex. Everybody is for it, (under certain conditions, of course). Everyone feels that they understand it, (even though they wouldn’t want to explain it.) Everyone thinks execution is only a matter of following natural inclinations (after all, we do get along somehow.) And, of course, most people feel that problems in these areas are caused by other people (if only they would take the time to do it right.)”

Crosby 1979

Definitions of Quality

uQuality is “fitness for use” - Juran

vthose product features which meet the needs of the

customers and thereby provide product satisfaction

vfreedom from deficiencies

uQuality is “conformance to requirements” and “zero defects” - Crosby

uQuality is “the totality of characteristics that bear upon its ability to satisfy stated or implied needs” - ISO9000

vstated needs - specified as requirements by a customer vimplied needs - identified and defined by the company

(3)

Defining Software Quality

uSoftware Quality is conformance to:

vexplicitly stated functional and performance

requirements

vexplicitly documented development standards vimplicit characteristics that are expected of all

professionally developed software

uSoftware Quality is ambiguous, subjective and multidimensional

The need for more detailed

measures of quality

uPrevious definitions are high-level and difficult to measure

uRequirements are difficult to specify

uRequirements are usually incomplete

uExperiment: Compare software warranties with hardware warranties

(4)

The Hardware Warranty

Company X warrants that the products it manufactures and sells are free from defects in materials and workmanship. If any product fails to operate properly, the company will repair the defective product and restore it to normal operation without charge

The Software Warranty

Company X’s sole obligation under this warranty will be to provide support

services described in our current software support policy. The company does not warrant that the licensed software is free from defects or that the support services will correct any defects that might exist.

(5)

McCall’s Software Quality Factors

Product operations Product revision Product transition Maintainability Flexibility Testability Portability Reusability Interoperability Correctness Reliability Efficiency Integrity Usability

The ‘ilities’ -Software Quality

Attributes - Product Operation

u Correctness

v Does it do what I want? v Tool - Use Cases u Reliability

v Does it do it accurately all of the time? v Tool - Formal methods

u Efficiency

v Will it run on my hardware as well as it can?

v Tool – Good algorithmic design, appropriate language (even

Assembler in some cases)

u Integrity v Is it secure? v Tool - Java u Useability

v Is it designed for the user? v Tool - User-centred design

(6)

The ‘ilities’ -Software Quality

Attributes - Product Revision

uMaintainability

vCan I fix it?

vTool - Encapsulation uFlexibility

vCan I change it? vTool - Encapsulation uTestability

vCan I test it? vTool - Interfaces

The ‘ilities’ -Software Quality

Attributes - Product Transition

uPortability

vWill I be able to use it on another machine?

vTool - Java, ANSI C, (and other emerging technologies) uReusability

vWill I be able to reuse some of the software?

vTool - OO class libraries (e.g. GUI components), function

libraries (e.g. Numerical Recipes in C)

uInteroperability

vWill I be able to interface it with another system? vTool - CORBA, DCOM

(7)

Metrics for Quality

uMany quality factors are difficult or impossible to measure directly

vNeed to develop indicative measures of the quality factors vUnfortunately many of these measures are still subjective uTwo methods are:

vMcCall’s checklist approach (see downloadon website)

vHewlett-Packard’s FURPS

v Functionality

» feature set, capabilities of program, generality of delivered functions, security of overall system

v Useability

» human factors, overall aesthetics, consistency and documentation v Reliability

» frequency and severity of failure, accuracy of output, MTTF, ability to recover from failure, predictability

v Performance

» processing speed, response time, resource consumption, throughput, efficiency v Supportability

» extensibility, adaptability, serviceability… testability, compat ibility, configurability, ease of installation, ease of problem localization

Software Quality Activities

uQuality Assurance

vThe production of organisational procedures and

standards which lead to high-quality software

uQuality Planning

vChoosing appropriate procedures and standards and

tailoring them for a specific software project

uQuality Control

vEnsuring that procedures and standards are followed by

(8)

Software Quality Assurance

uHow an organisation defines the methods by which it will achieve quality

uOrganisation must define or select standards

uQuality of the product is influenced by the quality of the process

uStandards need to be embedded in the software development process

uAll this will be documented in a Quality Manual

uThe relationship between software process and product quality is complex and poorly understood

Assuring Quality

uNeed to ask four questions

uWHAT attributes of the product manifest quality in your context?

uHOW is quality to be measured?

uWHEN do we evaluate the product and the process?

uWHO is responsible for carrying out the process?

(9)

Quality Principles

uTry to prevent defects from being introduced

uEnsure that defects that get in are detected and corrected as early as possible

uEstablish and eliminate the causes as well as the symptoms of defects

uMeasure quality characteristics

uIndependently audit work for compliance with standards and procedures

The Quality Plan

uIs specific to a project

uProduced early in the life of a project

uShould set out desired product qualities

uShould define how the quality is to be assessed

uShould indicate which standards are to be applied

uIndicate how compliance to the standards is monitored and assured

(10)

Contents of Software Quality Plan

from ISO9000

u Management responsibility u Quality system u Contract review u Design control u Quality control u Purchasing

u Customer supplied info

u Configuration management

u Process control

u Inspection and testing

u Inspection and testing equipment

u Control of non-conforming product

u Corrective action

u Handling, storage, packing and delivery

u Quality records

u Internal quality audits

u Training u Software maintenance u Statistical techniques u Control of the development environment

Quality Control

uEnsuring that all of the above gets done!

uTyranny is not effective in ensuring that the work is done

uPeople must see a clear benefit to them in performing the above activities

uEmbedding the procedures in day-to-day work is essential

uReviews are one of the major tools for quality control

(11)

Quality Standards

u Major standards used in Australia

u ISO9000

v international set of quality standards

v ISO9000-3:1997 is a specific subset aimed at software

development. It may be superceded by the latest IS9000 standard

u HB 90.9-2000

v Provides guidelines for the software industry on quality

management systems (QMS) complying with ISO9001:2000 (Quality Management Systems – requirements)

v Specifies requirements for a QMS for a software development

organization that

» “needs to demonstrate its ability to consistently provide product that meets customer and applicable regulatory requirements, and

» aims to enhance customer satisfaction through the effective application of the system, including processes for continual improvement of the system and the assurance of conformity to customer and applicable regulatory requirements.”

v Governments and other large organizations often require quality

certification, such as ISO9000

v earlier standards superceded by this include AS 3563 and

AS 3905.8

Quality Standards (2)

uOrganisations must be certified to be granted

ISO9000status

uCertification is granted by independent auditing groups (e.g. Standards Australia)

uCertification is not cheap (approximately $500,000 for a medium-sized company)

uCertification might not bring any benefits to the company in terms of quality, but could in terms of marketing

uCertification is an on-going process; audits are carried out annually

(12)

Quality Standards (3)

uTo get benefits from quality standards

vmust use sound management techniques vmust aim to improve the process

vemployees must participate actively

uQuality standards are of greatest value to those organisations which don’t already have formal development processes

uThey don’t replace individual skills and abilities

uThey can only be as good as the work practices which they document

Beyond Quality Standards

uQuality standards are only a necessary first step towards a software development

environment which produces quality software

uWe need to define what sub-processes are necessary in the overall process

uThe Capability Maturity Model (CMM) documents this for organisations

uThe Personal Software Process (PSP) does this for individual developers

(13)

The Capability Maturity Model

uDeveloped by the Software Engineering Institute (SEI), based on work by Watts Humphrey u5 Levels vLevel 1: Initial vLevel 2: Repeatable vLevel 3: Defined vLevel 4: Managed vLevel 5: Optimising

uAt each level Key Process Areas (KPAs) provide goals and example practices

Process Maturity Levels

Initial Optimising Managed Defined Repeatable Basic Management Process Definition Process measurement Process Control

(14)

Level 1: Initial

uProcess is ad hoc and sometimes chaotic

uFew processes are defined

uSuccess depends upon individual effort

uKey Process Areas

vNone

uToo many software development organisations are in this category

Level 2: Repeatable

uBasic project management processes defined

uProcess discipline in place to repeat successes

uChange is inherently dangerous at this level

uKey Process Areas

vConfiguration management vQuality assurance

vSub-contract management vProject tracking

(15)

Level 3: Defined

uProcess is documented, standardised and integrated into a standard software process

uAll projects use this standard software process

uMeasurement is still qualitative

uKey Process Areas

vPeer reviews

vIntergroup coordination vSoftware product engineering vIntegrated software management vTraining program

vSoftware process definition and focus

Level 4: Managed

uDetailed measures of software process and product quality are collected

uProcess and product are quantitatively controlled

uData gathering should be automated

uKey Process Areas

vQuality management

(16)

Level 5: Optimising

uContinuous process improvement is enabled

uThere is quantitative feedback from the process and from piloting innovative ideas and technologies

uMove from a purely product focus to focusing on process as well

uKey Process Areas

vProcess change management vTechnology change management vDefect prevention

The Personal Software Process

uMirrors the CMM for an individual

udesigned to help you become a better software engineer

urequires research, motivation and study to work

uFramework for

vwhy you make errors and how you find them vdetermining the quality of your reviews vdetermining the types of errors you make uDeveloped by Watts Humphrey

(17)

PSP0

uPSP0 - Baseline Process

vyour current process with some basic measurements and

a reporting format

vtime recording vdefect recording vdefect type standard uPSP0.1

va coding standard vsize measurement

vprocess improvement proposal

PSP1

uPSP1.0 - Personal Planning Process

vadds planning steps to PSP0: » test report

» size and resource estimation uPSP1.1

vto establish a performance rate for future planning vPSP1.0 enhanced by adding:

» task planning » schedule planning

(18)

PSP2

uPSP2.0 Personal Quality Management Process

vadds review techniques to PSP1 to help you find defects

early:

» design reviews » code reviews

» defect rates are typically 1 per 5-12 lines of code, do

you know what yours is?

uPSP2.1

vestablishes design completeness criteria: » design templates

PSP3

uPSP3 - Cyclic Personal Process

uFor large programs - 10,000 LOC

uSub-divide into PSP2-sized modules

uEnhance the base module in iterative cycles

uIn each iteration do a complete PSP2 including design, code, compile, test

uEffectively scale up from base module to large program, if each increment is of high quality

(19)

Overview of CMM and PSP

uCMM sets out the principal practices for managing the processes in large-scale software development

uPSP sets out the principal practices for defining, measuring and analysing an individual’s own processes

References

uPressman, Roger S., Software Engineering: A Practitioner’s Approach, McGraw-Hill, 2000 (Ch. 19).

uCapability Maturity Model® for Software (SW-CMM®), The Software Engineering Institute (SEI), Carnegie Mellon University.

http://www.sei.cmu.edu/cmm/cmm.html

uThe Personal Software ProcessSM_(PSPSM_),

The Software Engineering Institute (SEI), Carnegie Mellon University.