Lecture 20

Software Quality Engineering

BS(SE)-VI

Dr. Assad Abbas

Department of Computer Science

Topics

n

Quality Test Cases

Introduction

n

In the past, testing (other than some unit testing) was

undertaken as the last part of development after the entire

application was developed and documented.

n

Later, when problems were discovered too late, unit testing

and integration testing became in vogue.

n

As of 1994, some 24% of

project development

budget

was allocated to testing; 32% of

project management

budget slated for testing.

n

Major emphases on increased testing has met resistance

5

We cannot test everything!!!

Introduction…

5

Testing has become a huge business and major

emphasis in software engineering

n

Testing has been described as the process of

executing a program with the intention of finding

errors.

Software Testing - Definition

n

Software testing is a

formal

process carried out by a

Software Testing Objectives

n

Direct objectives

a)

To identify and reveal

as many errors as possible

in

the tested software.

b)

To bring the tested software, after correction of the

identified errors and retesting,

to an acceptable level

of quality

.

c)

To perform the required tests efficiently and

effectively, within the

budgetary

and

scheduling

limits

.

n

Indirect objectives

Software Testing Objectives

n

Incremental testing strategies:

5 Test incrementally: Unit testing; Integration testing; System testing

g

Bottom-up testing

Top-down testing

n

Big bang testing

Bottom-up Testing

M9

M8

M1 M2 M3 M4 M5 M6 M7

Bottom-up Testing

M9

M8

M1 M2 M3 M4 M5 M6 M7

M10 M11

Integration A

Integration B Integration c

Stage 2 Stage 4

Stage 3

Top-down Testing

M9

M8

M1 M2

M3 M4 M5

Some Observations

n

It is

impossible

to completely test any nontrivial module

or any system

5 Theoretical limitations: Halting problem

5 Practical limitations: Prohibitive in time and cost

loop 200 times

Test Plan

n

The goal of test planning is to establish the

list of tasks to be

performed

n

Testing will identify all of the requirements that have not been

met in the software.

n

The main work product is the test plan.

5

The test plan documents the overall approach to the

test. In many ways, the test plan serves as a

summary of the test activities that will be performed.

5

It shows how the tests will be organized, and outlines

all of the testers’ needs which must be met in order to

properly carry out the test.

Test Plan…

n

What

: a document describing the scope, approach,

resources and schedule of intended testing

activities; identifies test items, the features to be

tested, the testing tasks, who will do each task and

any risks requiring contingency planning.

n

Who:

QA.

Test Plan…

n

Why:

5 Divide responsibilities between teams involved; if more than one QA team is involved

5 Plan for test resources / timelines. 5 Plan for test coverage.

5 Plan for OS / DB / software deployment and configuration models coverage.

n

QA role:

5 Create and maintain the document. 5 Analyze for completeness.

Test Planning

• A Test Plan:

• covers all types and phases of testing.

• guides the entire testing process who, why, when, what

• developed as requirements,

functional specification, and high-level design are developed.

• should be done before implementation starts.

¨

A test plan includes:

 test objectives

 schedule and logistics

 test strategies

 test cases

 procedure  data

 expected result

Test Case

n

What:

a set of inputs, execution preconditions and

expected outcomes developed for a particular

objective, such as exercising a particular program

path or verifying compliance with a specific

requirement;

n

Who: QA;

n

When:

(planning)/(design)/coding/testing stage(s);

n

Why:

5 Plan test effort / resources / timelines; 5 Plan / review test coverage;

5 Track test execution progress; 5 Track defects;

Test Case…

n A test case is a description of a specific interaction that a tester will have in order to test

a single behavior of the software.

n Test cases are very similar to use cases, in that they are step-by-step narratives which

define a specific interaction between the user and the software.

n A typical test case is laid out in a table, and includes:

5 ID: unique identifier of a test case, i.e. a unique name and number.

5 Features to be tested / steps / input values – what you need to do

g A requirement which this test case is exercising.

5 Preconditions which describe the state of the software before the test case (which is often a previous test case that must always be run before the current test case)

5 Steps that describe the specific steps which make up the interaction

5 Expected Results / output values -what you are supposed to get from application

g which describe the expected state of the software after the test case is executed.

n Test cases must be repeatable.

Test Case…

–

Inputs

:

• Through the UI

• From interfacing systems or devices

• Files

• Databases

• State

• Environment

–

Outputs

:

• To UI

• To interfacing systems or devices

• Files

• Databases

Test Case Design

n

Generally, equivalence class testing is used to measure

effectiveness of tests. (Black Box testing)

n

Frequently, there are formulas included in the specifications.

Many of these have ‘categories’ of values to be tested.

5 Example:

–

Houses < $100,000

–

House > $100, <= $300,000

–

Houses > $300,000 and <= $750,000

Houses > $750,000

–

Such that each of these have formulas for computing,

Test Suite

n

A document specifying a sequence of actions for the

execution of multiple test cases.

n

Purpose

: to put the test cases into an executable order,

although individual test cases may have an internal set of

steps or procedures.

n

Is typically manual, if automated, typically referred to as

test script (though manual procedures can also be a type

of script).

n

Multiple Test Suites need to be organized into some

Test Execution

n The software testers begin executing the test plan after the

programmers deliver the alpha build, or a build that they feel is complete.

5 The alpha should be of high quality—the programmers should feel that it is ready for release, and as good as they can get it.

n There are typically several iterations of test execution.

5

The first iteration focuses on new functionality that has been

added since the last round of testing.

5

A

regression test

is a test designed to make sure that a

change to one area of the software has not caused any other

part of the software which had previously passed its tests to

stop working.

Defect Tracking

n

The defect tracking system

is a program that testers use to

record and track defects. It routes each defect between

testers, developers, the project manager and others, following

a workflow designed to ensure that the defect is verified and

repaired.

5 Every defect encountered in the test run is recorded and entered into a defect tracking system so that it can be prioritized.

Postmortem Reports

n

The

postmortem report

is an overall account of the

team’s experience in building the software, and of

the experience of the users and stakeholders in

working with the team.

5

The report should contain an honest assessment of

how the team members, users, and stakeholders

perceived the end product, and assessed the decisions

made throughout the project.

Software Test Classifications

n While quite different (each has strong proponents), there are two

accepted classification schemes: 5 Black Box testing

5 While Box testing.

n

Black box testing – IEEE definition

5 Testing that ignores the internal mechanism of the system or

component and focuses solely on outputs in response to selected inputs and execution conditions.

5 Testing conducted to evaluate the compliance of a system or component with specified functional requirements.

n White box testing – IEEE definition

Black-box Testing

n

Focus:

I/O behavior

. If for any given input, we can

predict the output, then the module passes the test.

5

Almost always impossible to generate all possible

inputs ("test cases").

n

Goal:

Reduce number of test cases

by equivalence

partitioning:

5

Divide input conditions into equivalence classes.

5

Choose test cases for each equivalence class.

Black-box Testing…

n

Selection of equivalence classes (

No

rules, only guidelines):

5

Input is valid across range of values. Select test cases

from 3 equivalence classes:

g Below the range g Within the range g Above the range

5

Input is valid if it is from a discrete set. Select test cases

from 2 equivalence classes:

g Valid discrete value g Invalid discrete value

n

Another solution is to select only a limited amount of test

cases:

White-box Testing

n

Focus: Thoroughness (

Coverage

). Every statement in the

component is executed at least once.

n

Four types of white-box testing

5

Statement Testing

5

Loop Testing

5

Path Testing

White-box Testing…

n Statement Testing (Algebraic Testing): Test single statements

n Loop Testing:

5 Cause execution of the loop to be skipped completely. (Exception: Repeat loops)

5 Loop to be executed exactly once

5 Loop to be executed more than once

n Path testing:

5 Make sure all paths in the program are executed

n Branch Testing (Conditional Testing): Make sure that each possible outcome

from a condition is tested at least once.

White-box Testing- Loop Testing

Nested Loops

Concatenated Simple

White-box Testing: Example

C = A + B IF C>100

PRINT “ITS DONE” END IF

IF A>50

White-box Testing: Example

n Red Line TestCase_01: A=50, B=60 n Blue Line TestCase_02: A=55, B=40 n Orange Line TestCase_03: A=40,

B=65

n Green Line TestCase_04: A=30,

White-box Testing: Example

/*Read in and sum the scores*/ FindMean(float Mean, FILE ScoreFile)

{ SumOfScores = 0.0; NumberOfScores = 0; Mean = 0; Read(ScoreFile, Score);

while (! EOF(ScoreFile) { if ( Score > 0.0 ) {

SumOfScores = SumOfScores + Score; NumberOfScores++;

}

Read(ScoreFile, Score); }

/* Compute the mean and print the result */ if (NumberOfScores > 0 ) {

Mean = SumOfScores/NumberOfScores;

printf("The mean score is %f \n", Mean); } else

09/22/20201 2 3 4 5 7 6 8

FindMean (FILE ScoreFile) { float SumOfScores = 0.0;

int NumberOfScores = 0;

float Mean=0.0; float Score; Read(ScoreFile, Score);

while (! EOF(ScoreFile) { if (Score > 0.0 ) {

SumOfScores = SumOfScores + Score; NumberOfScores++;

}

Read(ScoreFile, Score); }

/* Compute the mean and print the result */ if (NumberOfScores > 0) {

Mean = SumOfScores / NumberOfScores;

Constructing the Logic Flow Diagram

Start

2

3

4 5

6

7

8 9

Exit 1

F

T F

T F

Test Cases

a (Covered by any data)

(Data set must

(Data set must contain at least one value

be empty)

(Total score > 0.0) (Total score < 0.0)

(Positive score) _{(Negative score)}

Example: MPT

n Consider the basic annual municipal property tax on apartments.

The basic municipal property tax (before discounts to special groups of city dwellers) is based on the following parameters:

5 S, the size of the apartment (in square yards)

5 N, the number of persons living in the apartment

5 A, B or C, the suburb’s socio-economic classification.

n The municipal property tax (MPT) is calculated as follows:

5 For class A suburbs: MPT = (100 × S) / (N + 8)

5 For class B suburbs MPT = (80 × S) / (N + 8)

5 For class C suburbs MPT = (50 × S) / (N + 8)

References

1. Software Quality Engineering Testing, Quality Assurance, and

Quantifiable Improvement, Jeff Tian, Chapter 14 and 15, Wiley, 2005.

2. Software quality assurance: from theory to implementation. Daniel