Softenr-10

Software Engineering

Software Testing

Chapter-14: Testing

Objectives

— To understand the role of testing in ensuring software quality

—To discuss issues relevant to software testing —To describe the different techniques used for

The Process of Coding, Testing

and Installation

• _Coding

– _{Physical design specifications are turned into}

working computer code.

• _Testing

– _{Tests are performed using various strategies.}

– _{Testing can be performed in parallel with}

coding.

• _Installation

– _{The current system is replaced by the new}

4

Validation & Verification

• _Validation

– _{“Are we building the product right?”}

– _{Ensure software meets customer’s needs}

• _Verification

– _{“Are we building the right product?”}

6

Why We Need Software Testing?

• _{Reveal faults/failures/errors} • _{Locate faults/failures/errors} • _{Show system correctness}

• _{Improved confidence that system performs as} specified (verification)

• _{Improved confidence that system performs as} desired (validation)

Importance of Testing

• _{Critical Element of Software Quality Assurance} • _{Well-planned and thorough testing are}

important in reducing the high cost of software failure

• _{Can take up to 40% of development effort}

• _{In systems that require high reliability, testing}

8

Who Should Test The Software?

• _{Developer (individual units)}

• _{Independent test group (ITG)}

– _{removes conflict of interest}

User Acceptance Testing

• _{Actual users test a completed information}

system.

• _{End result is the users’ final acceptance of the}

system.

• _{Alpha testing: use simulated data}

• _{Beta testing: use real data in real user}

Test Cases

• _{Test case: a scenario of transactions,}

queries or navigation paths

• _{Can represent either:}

– Typical system use

– _{Critical system use}

– _{Abnormal system use}

• _{Test cases and results should be thoroughly}

Test Cases (cont)

• _{Test case : unit of testing activity} • _{Test cases have 3 parts}

:-– _Goal

• _{Aspect of the system being tested}

– _{Input and system state}

• Data provided to the system under stated condition

– _{Expected behavior}

Unit testing

• _{Unit testing is the process of testing individual}

components in isolation.

• _{It is a defect testing process.} • _{Units may be:}

– _{Individual functions or methods within an object} – _{Object classes with several attributes and methods} – _{Composite components with defined interfaces}

Manual Testing Techniques

• _Inspection

– A testing technique in which participants examine program code for predictable language-specific errors • Walkthrough

– _{A peer group review of any product created during}

the systems development process; also called a structured walkthrough

• Desk Checking

– _{A testing technique in which the program code is}

White Box Testing

• _{Derived from knowledge of program’s}

structure & implementation

• _{Structural testing - analyse code & use}

knowledge of the structure of a component to derive test data

• _{Logical paths are tested by providing test}

18

White Box Testing

• _{Thorough white box testing would lead to}

“100 percent correct programs”

• _{Exhaustive testing are impractical - too many}

tests!

• _{A limited number of logical paths can be}

selected and exercised

• _{Important data structures can be probed for}

White Box Test Cases

• _{Guarantee that all independent paths have}

been exercised at least once

• _{Exercise all logical decisions on their true and}

false sides

• _{Execute all loops at their boundaries and}

within their operational bounds

• _{Exercise internal data structures to ensure}

Decision Coverage

• _{Decision (Branch) Coverage Method}

• _{Causes every decision (if, switch, while, etc.) in the}

program to be made both ways (or every possible way for switch)

• _{System: Design a test case to exercise each decision in}

the program each way (true / false)

• _{Completion criterion: A test case for each side of each}

decision

Condition Coverage

• _{Condition Coverage Method}

• _{Like decision coverage, but causes every condition expression to be}

exercised both ways (true / false)

• _{A condition is any true / false subexpression in a decision}

– Example:

• _{if (( x == 1 || y > 2) && z < 3)}

• _{Requires separate condition coverage tests for each of:}

• _{x == 1 true / false} • _{y > 2 true / false} • _{z < 3 true / false}

• _{More effective than simple decision coverage since exercises the}

Loop Coverage Method

• _{Most programs do their real work in do, while and for loops}

• _{This method makes tests to exercise each loop in the program in four different states} – _{execute body zero times (do not enter loop)}

– _{execute body once (i.e., do not repeat)} – _{execute body twice (i.e., repeat once)} – _{execute body many times}

• _{Usually used as an enhancement of a statement, block, decision or condition coverage}

method

• _{System: Devise test cases to exercise each loop with zero, one, two and many}

repetitions

Loop Coverage Method

if there is an upper bound, n, on the number of times the loop body can be executed, then the following cases should also be applied.

•_{Design a test in which the loop body is executed exactly n-1}

times.

•_{Design a test in which the loop body is executed}

exactly n times.

•_{Try to design a test causing the loop body to be executed}

Loop Coverage Method

Apply the following guidelines to ensure that each loop is tested at its boundaries (that is, with both minimal and maximal numbers of iterations of the loop body) while using a number of tests that is only linear in the number of loops:

•Conduct the ``simple loop tests'' for the innermost loop (which is a simple loop), while keeping the number of iterations of the outer loops at their minimal

nonzero values.

•Work outward, conducting tests (as described above) for each loop, while holding the number of iterations of outer loops at the minimal nonzero values possible (that is, the minimal values that can be used when the inner loop body is to be executed the desired number of times), and holding the number of

Execution Paths

• _{An execution path is a sequence of executed}

statements starting at the entry to the unit (usually the first statement) and ending at the exit from the unit (usually the last statement)

• _{Two paths are independent if there is at least one}

statement on one path which is not executed on the other

• _{Path analysis (also know as cyclomatic complexity}

28

Flow Graph Notation

Converting Code to Graph

if expression1 then

statement2

else

statement3 end if

statement4

switch expr1 case 1:

statement2 case 2:

statm3 case 3: statm4 end switch statm5 (a) (b) do statement1

CODE FLOWCHART GRAPH

T _expr1 F ?

statm4

statm2 statm3

2

1 _expr1 3 ? statm5 statm3 statm2 statm4 n1 n2 n3 n4 n1 n2 n4 n5 n3 statm1 _n1

For a strongly connected graph: Create a virtual edge

Cyclomatic Complexity

• _{Region, R= 6}

Number of Nodes = 13 Number of edges = 17

Cyclomatic Complexity

• _{Cyclomatic Complexity for a flow graph is computed in one of three ways:} • The numbers of regions of the flow graph correspond to the Cyclomatic

complexity.

• Cyclomatic complexity, V(G), for a flow graph G is defined as

V(G) = E – N + 2

where E is the number of flow graph edges and N is the number of flow graph nodes.

• _{Cyclomatic complexity, V(G), for a graph flow G is also defined as}

Cyclomatic Complexity

Cyclomatic Complexity, V( C) : V( C ) = R = 6;

Or

V(C) = Predicate Nodes + 1 =5+1 =6

Or

Execution Paths Analysis

• _{To achieve 100% basis path coverage, you}

need to define your basis set. The cyclomatic complexity of this method is four (one plus the number of decisions), so you need to

define four linearly independent paths. To do this, you pick an arbitrary first path as a

Execution Paths Analysis

• _{Path 1: Any path will do for your baseline, so pick true for the decisions'}

outcomes (represented as TTT). This is the first path in your basis set.

Path 2: To find the next basis path, flip the first decision (only) in your baseline, giving you FTT for your desired decision outcomes.

Path 3: You flip the second decision in your baseline path, giving you TFT for your third basis path. In this case, the first baseline decision remains fixed with the true outcome.

Path 4: Finally, you flip the third decision in your baseline path, giving you TTF for your fourth basis path. In this case, the first baseline decision

remains fixed with the true outcome.

Path Coverage Testing

• _Advantages

– Covers all basic blocks (does all of basic block testing)

– _{Covers all conditions (does all of decision/condition}

testing)

– Does all of both, but with fewer tests!

– _{Automatable (in practice requires automation)}

• _{Disadvantages}

if expression1 then

statement2 end if

do

statement3

while expr4

end do

if expression5 then

statement6 end if

statement7

Solution

if expression1 then

statement2 end if

do

statement3

while expr4

end do

if expression5 then

statement6 end if

statement7

Discussion on White Box Testing

• _Advantages

– _{Find errors on code level}

– _{Typically based on a very systematic approach, covering}

the complete internal module structure

• _Constraints

– _{Does not find missing or additional functionality} – _{Does not really check the interface}

40

Black Box Testing

• _{Derived from program specification}

• _{Functional testing of a component of a system}

• _{Examine behaviour through inputs & the}

corresponding outputs

• _{Input is properly accepted, output is correctly}

produced

Black Box Testing

Attempts to find the following errors:

• _{Incorrect or missing functions} • _{Interface errors}

• _{Errors in data structures or external database}

access

• _{Performance errors}

Types

Types of Black Box Testing

There are many types of Black Box Testing but following are the prominent ones

-Functional testing - This black box testing type is related to functional requirements of a system; it is done by software testers.

Non-functional testing - This type of black box testing is not related to testing of a specific functionality , but non-functional requirements such as performance, scalability, usability.

Equivalence Partitioning

• _{Divide input domain into classes of data}

• _{Based on an evaluation of equivalence classes for an} input condition

• _{Guidelines to define equivalence classes}

– _{Range input : One valid and two invalid equivalence} – _{Specific value : One valid and two invalid equivalence}

44

Example – Data for Automated Banking Application

The use can access the bank using his personal computer, provide a six digit password, and follow with a series of typed commands that trigger various banking function. During the log on sequence the software supplied for the banking application accepts data in the form:

area code – blank or 3 digit numbers

prefix – 3 digit numbers, nor beginning with 0 or 1 suffix – 4 digit numbers

password – 6 digits alphanumeric strings commands – “check”, “deposit”, “bill pay” etc

Input condition

area code : Input condition : Boolean – area code may or may not present Input condition : Range – 200 – 999 with specific exception prefix : Input condition : Range – specified value > 200 with no 0 digits suffix : Input condition : Value – 4 digit length

password : Input condition : Boolean – password may or may not present Input condition : Value – six character string

46

Boundary Value Analysis

• _{Complement equivalence partitioning} • _{Test both sides of each boundary}

• _{Look at output boundaries for test cases}

• _{Test min, min-1, max, max+1, typical values} • _{Example : 1 <= x <=100}

48

Comparison Testing

• _{Used only in situations in which the reliability of}

software is absolutely critical (e.g., human-rated systems)

– _{Separate software engineering teams develop}

independent versions of an application using the same specification

– _{Each version can be tested with the same test data to}

ensure that all provide identical output

– _{Then all versions are executed in parallel with}

Orthogonal Array Testing

• _{Used when the number of input parameters}

is small and the values that each of the parameters may take are clearly bounded

X Y

Z

X Y

50

Discussion on Black Box Testing

• _Advantages

– _{Find missing functionality}

– _{Independent from code size and functionality} – _{Find some coding errors}

• _Constraints

Black box testing techniques

Black box testing strategy:

Following are the prominent test strategy amongst the many used in Black box Testing

Equivalence Class Testing: It is used to minimize the number of

possible test cases to an optimum level while maintains reasonable test coverage.

Boundary Value Testing: Boundary value testing is focused on the values at boundaries. This technique determines whether a certain

range of values are acceptable by the system or not. It is very useful in reducing the number of test cases. It is mostly suitable for the systems where input is within certain ranges.

Summary

We have learned

• To understand the role of testing in ensuring software quality

References

1- These slides are designed to accompany Software

Engineering: A Practitioner’s Approach, 7/e (McGraw-Hill 2009). Slides copyright 2009 by Roger Pressman.

2- http://www.cs.unc.edu/~stotts/145/cocomo7.gif

3-Software Engineering” by Ian Somerville, Addison-Wesley, 2009