CS 5154: Software Testing. Input Space Partitioning

(1)

CS 5154: Software Testing Input Space Partitioning

Instructor: Owolabi Legunsen

Fall 2021

(2)

How many inputs can we test this method on?

/** Compute the arithmetic mean of A, B, and C.

* Assume a 32-bit computer

**/

private static double computeAverage (int A, int B, int C) {

… }

= 2^32 x 2^32 x 2^32

= 79.23 x 10²⁷

≅ 80 Octillion

(3)

How many inputs should we test this method on?

/** Compute the arithmetic mean of A, B, and C.

* Assume a 32-bit computer

**/

private static double computeAverage (int A, int B, int C) {

… }

(4)

Input

Domains Graphs Logic

Expressions Syntax

A: {0, 1, >1}

B: {600, 700, 800}

C: {cs, ece, is, sds}

(not X or not Y) and A and B

if (x > y) z = x - y;

else

z = 2 * x;

Let’s see how model-driven test design can help

Recall that we will look at four models in this course

(5)

ISP: choose inputs from the input domain

• You have been choosing inputs for programs all your career

• ISP helps you

• choose those inputs more systematically

• check that inputs come from different parts of the input domain

• know when to stop choosing

(6)

3 key concepts in ISP

1. Input Domain: the set of all possible inputs to a program 2. Input Parameters that a program takes

• e.g., method arguments, user inputs, file/database data, global variables

• parameters define the scope of the input domain

3. Input Domain Modeling: process of reasoning about the input domain

(7)

The core activities in Input Domain Modeling

1. Partition the input domain into regions called blocks

2. Choose at least one value for each block

3. Combine values across various parameters

Wait… that’s it?!

(8)

Partitioning the input domain into blocks

• Decide on characteristics of your input domain to partitioning on

• Assumption: values in each block are equally useful for testing

• Example: Program: void foo(String char) // “char” is a letter Input domain: Alphabetic letters

Partitioning characteristic: Case of letter

• Block 1: upper case

• Block 2: lower case

(9)

Your turn: examples on partitioning

• Program: void foo(String char) // “char” is a letter

• Input domain: Alphabetic letters

• Partitioning characteristic:

• Block 1:

• Block 2:

• Block 3:

(10)

Your turn: examples on partitioning (2)

• Program: void foo(String char) // “char” is a letter

• Input domain: Alphabetic letters

• Block 1:

• Block 2:

• Block 3:

(11)

Your turn: examples on partitioning (3)

• Program: void bar(List<String> l)

• Input domain:

• Block 1:

• Block 2:

• Block 3:

(12)

Your turn: examples on partitioning (4)

• Program: void baz(List<String> filenames)

• Input domain:

• Block 1:

• Block 2:

• Block 3:

(13)

How to know that partitioning is “correct”?

• Let the input domain be D

• Characteristic q of D defines a set of blocks, B_q= {b₁,b₂,…,b_Q}

• Each partition must satisfy two properties :

1. Blocks must be pairwise disjoint (no overlap)

2. Together the blocks must cover the domain D (complete)

b

_i

∩ b

_j

= ∅,  i  j, b

_i

, b

_j

 B

_q

⋃

^{b = D}

b  B_q

b₁ b₂ b₃

(14)

Partitioning is simple but easy to do wrong

• Consider the characteristic “order of elements in list F”

b₁ = sorted in ascending order b₂ = sorted in descending order b₃ = arbitrary order

but … something’s fishy …

What if the list is of length 0 or 1?

The list will be in all three blocks That is, disjointness is not satisfied

Solution:

Two characteristics that each address just one property

C1: List F sorted ascending - c1.b1 = true

- c1.b2 = false

C2: List F sorted descending - c2.b1 = true

- c2.b2 = false

Design blocks for that characteristic

Can you find the problem?

Can you think of a solution?

(15)

Your turn: work with your neighbor in pairs

Make sure your partition is valid: (1) Pairwise disjoint, and (2) Complete Design a partitioning for the input domain: all integers

That is, partition integers into blocks such that each block seems to be equivalent in terms of testing

(16)

Discuss with a nearby pair

• What characteristics did you all come up with?

• Are the blocks pairwise disjoint and complete?

• Work together to fix any problems that you find with the partitioning

(17)

Characteristics, blocks, problems that you discussed?

(18)

Characteristics, blocks, problems that you discussed?

(19)

Questions so far?

?

(20)

Let’s reflect on what we learned

• Partitioning is a creative design activity

• many partitions can be created for the same input domain

• If done carelessly, partitions may not be complete or pairwise disjoint

• Consider different alternatives and the trade-offs that they induce

(21)

• Multiple partitioning schemes may exist for the same code

• Considering cost/benefit tradeoffs in designs is an essential part of SE

ISP is a design activity 

21

(22)

(1) How is any of these systematic?

(2) Why did you say that ISP is a way to do

Model-driven test design?

(23)

Recall: what we do in MDTD

• Move from implementation level to abstraction level

• At the abstraction level, define test requirements and find input values that satisfy them

• Back in the implementation level: write, run, and evaluate tests for the inputs from (2)

(24)

How do we implement these steps for ISP?

• Step 1: Identify testable functions in your program

• Step 2: Find all your input parameters

• Step 3: Model the input domain

• Step 4: Use a criterion to choose combination of values

• Step 5: Refine combinations of blocks into test inputs

(25)

The five ISP steps by example

• Consider method triang() from class TriangleType :

• http://www.cs.gmu.edu/~offutt/softwaretest/java/Triangle.java

• http://www.cs.gmu.edu/~offutt/softwaretest/java/TriangleType.java

public enum Triangle { Scalene, Isosceles, Equilateral, Invalid } /** side1, side2, and side3 are lengths of the sides of a triangle

* Returns the appropriate enum value

**/

public static Triangle triang (int side1, int side2, int side3)

(26)

Step 1: Identify testable functions in TriangleType

**/

(27)

Identifying testable functions more generally

• Individual methods have one testable function

• What if the method is private?

• What if a method calls other methods?

• Each method in a class should be tested individually

• But methods in a class may share characteristics that you can reuse

(28)

Step 2: Find input parameters for triang()

**/

(29)

Finding input parameters for testable functions

• Simple step, but be careful to identify all parameters

• Remember to check if program state is an input parameter

• Remember to check if data sources are input parameters

find(String key) // find location of key in a file add(E e) // add element e to Set

(30)

Step 3: Model the input domain for triang()

• Consider only parameter types or the functionality of triang()?

• How to combine values obtained from IDM of all parameters?

• What is the correct IDM for triang()?

public static Triangle triang(int side1, int side2, int side3)

(31)

Two approaches to IDM

• Interface-based: develop characteristics only from input parameters

• e.g., triang() takes three ints

• Functionality-based: use behavioral view to develop characteristics

• e.g., triang() returns a Triangle

• Which approach should we use?

(32)

Interface-based IDM: Example

• Input domain:

• Block 1:

• Block 2:

• Block 3:

/** side1, side2, and side3 are lengths of the sides of a triangle

**/

(33)

Interface-based IDM: Pros and Cons

 easy to identify characteristics and translate to test cases

 almost mechanical to follow

 may not encode all the information that we know

 can miss tests if functionality depends on combination of values

(34)

Functionality-based IDM: Example

• Input domain:

• Block 1:

• Block 2:

• Block 3:

**/

34

(35)

Functionality-based IDM: Pros and Cons

 allows incorporation of semantics or domain knowledge

 can be done earlier from requirement specifications

 harder to develop characteristics, e.g., large systems, missing specs

 can miss tests if functionality depends on combination of values

(36)

Poll: which approach should we use

• Interface-based

• Functionality-based

• Both

• None

(37)