Lecture 14

Software Quality Engineering

BS(SE)-VI

Dr. Assad Abbas

Department of Computer Science

2

9/22/20

Topics

n The elements of a quality assurance plan

n The contents of a quality assurance plan

n Quality plan for small and internal projects

Reality Check

n

Is an SQA plan just busy-work, or does it really

pay off?

n

Hughes Aircraft

5 moved from level 2 in 1987 to level 3 in 1990

5 cost = $500K

5 benefit = $2M annually

n

Raytheon

5 moved from level 1 in 1988 to level 3 in 1993

5 productivity doubled

4

9/22/20

CMM Effectiveness Case Studies

n

Hughes Aircraft

n

Schlumberger

n

Texas Instruments

Boehm and Ross’s Top 10 Software Risk Items

1. Developing wrong software functions * 2. Unrealistic schedules and budgets * 3. Developing wrong user interface

4. Gold plating *

5. Continuing stream of requirement changes * 6. Shortfalls in externally furnished components 7. Shortfalls in externally performed tasks

8. Personnel shortfalls *

6

9/22/20

The Elements of a Quality Plan

n

List of Quality Goals

n

Review Activities

n

Software Tests

n

Acceptance Tests for Software Externally

Developed

8

9/22/20

The Elements of a Quality Plan…

List of Quality Goals

n These refer to the quality requirements in the developed software system. n Quantitative measures usually preferred to qualitative measures when

choosing goals because they are usually easier to assess objectively during testing.

n Quality goals should reflect the major acceptance criteria found in the

Example

n A software system to serve the help desk operations of an

electrical appliance manufacturer is to be developed. The

help desk system (HDS) is intended to operate for 100 hours per week. The software quality assurance team was

requested to prepare a list of quantitative quality goals

appropriate to certain qualitative requirements.

n HDS qualitative requirement

5 The HDS should be user friendly.

n Related quantitative goals

10

9/22/20

09/22/2020

The Elements of a Quality Plan…

Planned Review Activities

n The planned reviews should include a complete listing of all planned

review activities.

5 Design reviews (DRs)

5 Technical reviews

5 Managerial reviews

5 Code inspections

5 Pros and Cons………

n All reviews need to include:

5 Scope – what does it cover

5 Type – emphasis – managerial, technical, super detailed…

5 Schedule – often based on previous reviews and outcomes

g As defined by its priority and the succeeding activities of the project process.

5 Procedures – action lists;

5 Who is to attend? Collateral interest?? *****

12

9/22/20

The Elements of a Quality Plan…

Planned Software Tests

n

The quality plan Must provide a

complete list

of

planned tests.

n Each test must include the following:

5 Coverage of test: unit, integration, system, subsystem….

5 Type of test: may include computer-generated tests and their application via test suites, and more

5 Planned test schedule – prioritized and follow up Exact procedures (for different types of tests…)

5 Who is responsible for carrying out tests

The Elements of a Quality Plan…

Planned Acceptance Tests for Externally Developed Software

n A complete set of acceptance tests to be run for externally developed

software must be provided within the quality plan!

n Complete set must be run for our own developed software!

n Especially critical for purchased software, contracted software (i.e.

software developed by subcontractors), customer-supplied software.

n These tests can be run in parallel with internally-developed software tests

14

9/22/20

The Elements of a Quality Plan…

n The quality plan MUST include configuration management tools and

procedures for managing the software configurations, versions, etc.

n Must be an intrinsic part of the entire project!

n The Quality Plan may be included within the Development Plan or as an

independent document.

n The document, however compiled, must be reviewed and approved by the

16

9/22/20

Sources of Changes

n Requirements

n Changes in funding

n Technology advancements n Solutions to problems

n Scheduling constraints n Customer expectations

n Unexpected opportunities for an improved system

Contents of a Quality Assurance Plan

n

Project

Details

5 Project Manager, Quality Manager 5 Project objectives

n

Project

Standards and Methodologies

For IT based projects these might be such things as: 5 Development Methodology (ies) being used

5 Documentation standards

5 Coding standards e.g. ANSI C 5 Testing standards and methods

5 QA methods e.g. walkthroughs or inspections 5 Operating standards e.g. ISO 9000, ISO 17799 5 Project Management Methodology e.g. PRINCE

18

9/22/20

Plan Contents (deliverables)

n

Project

Deliverables

5

Design specifications

5

Test specifications

5

Documentation

5

Software and Hardware

n

Quality

Goals

5

Quantitative and Qualitative quality goals that are to

be met by the deliverables

Plan Contents (deliverables)…

n

Quality Assurance

Schedule

5 Methodologies to be used at each stage of development (inspections, walkthroughs, tests etc.)

5 Test Plan reviews

5 Sampling methods to be used, e.g., on test results 5 Who will participate in reviews and inspections

n

Risks and Exposures

5 List of key risks to project

5 Actions to be taken to address them 5 Personnel responsible

20

9/22/20

Quality Control Points

Requirements Gathering System Design Inspection Subsystem Design Inspection Module Design Inspection

Write Source Code

Inspection

System Test Plans

Inspection

Test Cases

Inspection

System Test Cases

Module Test Plans

Inspection

Inspection

Unit Testing System Testing

Integration Test Plans

Integration Testing Inspection Inspection

Development / Quality Plans for Small and Internal

Projects

n Natural for many to try to avoid hassle of preparing all these

plans.

n In fact, heavy-weight methodologies are often called

plan-centric; Agile methods try to avoid much planning and documentation.

n The question is simply does a short, small project (likw 30-60

days; two or three individuals) deserve the time spent on planning a development and quality plan?

22

9/22/20

Development / Quality Plans for Small Projects

n Lots of issues here…

n Sometimes not done due to short duration / manpower n Sometimes planning is left up to the project leader’s

discretion.

n Perhaps a critically-important and high risk but short duration

effort with high-penalty shouts for a plan…

n Sometimes, via contract, both development and quality plans

Development / Quality Plans for Small Projects…

n Several advantages to planned over unplanned projects:

1. A more comprehensive / thorough understanding of the task is likely (gained when developing the plan).

2. Greater responsibility for meeting obligations can be assigned, as they can be ‘seen’ more clearly since articulated (who does what).

3. Easier to share control of the project and identify

unexpected delays (any plan better than no plan at all!)

24

9/22/20

Development / Quality Plans for Internal Projects

n Lots of projects done for the internal use of organization.

n Here, normally no external body is a customer n Can be medium or large scale.

n Tendency to avoid adequate development / quality plans

5

avoiding plans is fraught with errors.

Preparing Plans provides Advantages

n

Internal Customers ‘

can

’ enjoy advantages

5 smaller deviations from planned completion dates.

5 smaller budget overruns.

5 better control over development process – problems

can be addressed locally.

n

Organizationally,

5 reduced risk of market loss (done for internal use).

5 reduced risk of litigation (late arrival; non-compliance).

5 reduced risk of impairing a firm’s reputation.

26

9/22/20

References

1.

Software Quality Assurance: from Theory to