Testing, Debugging, and Verification

(1)

Testing, Debugging, and Verification

Testing, Part II

Moa Johansson

(2)

Admin

I Make sure you areregisteredfor the course. Otherwise your marks cannot be recorded.

I Even if you arerepeating the course, only taking exercises or exam.

I If in doubt, contact the student center to double check.

I PhD students excluded (but drop me an email so I know who you are)

I If anyone have a clashing exam: contact student center/exam administration.

I Please sign up to the News Google-group.

(3)

Exercise sessions

I First exercise session this Wednesday

I please bring laptops

I install relevant tools before

I topic: testing

I install JUnit beforehand

(4)

Overview of this Lecture

I Testing levels and the V-model

I Focus on Unit Testing

I Terminology: Test case, test set, test suit, oracle

I Introduction to JUnit: a framework for rapid unit testing

(5)

Recall: The V-Model

Integration Testing Unit Testing Implementation Subsystem Design Architectural Design Acceptance Testing System Testing Requirements Analysis Customer needs Choose components, connections Structure, behaviour of subsystem Code!

User's/client's needs met?

Assembled system meets spec?

Does components work together?

Test individual methods, classes

(6)

Examples: System, Unit and Integration Testing

Pentium Bug (-94)

I Wrong result on floating point divisions.

I Omission of 5 entries in 1066 entry table.

I Rarely happened (on system level).

I Easy catch inunit test. Arianne 5 Rocket (-96)

I Exploded 5 secs. after takeoff.

I Used guidance system from Arianne 4.

(7)

Examples: System, Unit and Integration Testing

Mars Lander (-99)

I Crashed on landing.

I Mismatch in units: imperial vs. metric.

(8)

Discussion: Testing Levels of a System for Printing

Paychecks

Name: Hours: Print Paycheck Payroll System calculates: salaries, taxes, bonus

Print Paycheck

Formats and prints paycheck

Staff Database

Given name, looks up address, position.

Person nr:

Think of examples of:

I System Tests

I Integration Tests

(9)

Some examples of Tests

I System Test

I Enter data in GUI, does it print the correct paycheck, formatted as

expected?

I Integration Tests, e.g.

I Payroll asks database for staff data, are values what’s expected?

Maybe there are special characters (unexpected!).

I Are paychecks formatted correctly for different kinds of printers?

I Unit Tests, e.g.

I Does payroll system compute correct tax-rate, bonus etc?

I Does the Print Paycheck button react when clicked?

(10)

Regression Testing

Orthogonal to the above testing levels: Regression Testing

I Testing that is doneafter changesin the software.

I Purpose:

gain confidence that the change(s) did not cause (new) failures.

I Standard part of the maintenance phase of software development. E.g. Suppose Payroll subsystem is updated. Need to re-run tests (which ones?).

(11)

Unit Testing

Rest of testing part of the course: focusing largely on unit testing recall: unit testing= procedure testing = (in oo) method testing major issues in unit testing:

1. unit test cases (‘test cases’in short) 2. order in which to test and integrate units start with 1.

(12)

Test Cases

(13)

What does a test case consists of?

(to be refined later)

Test case

I Initialisation (of class instance and input arguments)

I Call to the method under test.

I Decision (oracle) whether the test succeeds or fails

I two first parts seem enough for a test case,

(14)

‘Success’ vs. ‘Failure’ of Tests

What does it mean for a test tosucced? ... or fail?

(15)

Test Cases, more precise

Params,

Input State Final StateResult,

Method m I: (P, S_in) (R, S_out) Yes! No! Oracle O Oracle O More formally...

Atest caseis a tuple hm, I, Oi of methodm, inputI, and oracleO, where

I m is the method under test

I _{I is a tuple hP, S}_ini

ofcall parameters P andinitial stateSin

I O(R, Sout) 7→ {pass, fail }

is a functiononreturn valueR and final stateSout, telling whether

(16)

Test Set

Atest set TSm for a (Java) method m consists of n test cases: TSm = {hm, I1, O1i, . . . , hm, In, Oni}

(17)

Test Suite

Atest suite for methods m1, . . . , mk is a union of corresponding test sets:

(18)

Automated and Repeatable Testing

Basic idea: writecode that performs the tests.

I By using a tool you can automatically run a large collection of tests

I The testing code can be integrated into the actual code, thus stored in an organised way

I side-effect: documentation

I After debugging, the tests are rerunto check if failure is gone

I Whenever code is extended, all old test cases can be rerun to check that nothing is broken (regression testing)

(19)

Automated and Repeatable Testing (cont’d)

We will use JUnitfor writing and running the test cases. JUnit: small tool offering

I some functionality repeatedly needed when writing test cases

I a way to annotate methods as being test cases

(20)

JUnit

I _{Java testing framework to write and run automated tests} I JUnit features include:

I Assertions for testing expected results

I Annotations to designate test cases

I Sharing of common test data

I Graphical and textual test runners

I JUnit is widely used in industry

I JUnit used from command line or within an IDE (e.g., Eclipse) (Demo)

(21)

Basic JUnit usage

p u b l i c c l a s s Ex1 { p u b l i c s t a t i c i n t f i n d _ m i n ( i n t [] a ) { i n t x , i ; x = a [ 0 ] ; f o r ( i = 1; i < a . length ; i ++) { i f ( a [ i ] < x ) x = a [ i ]; } return x ; } ...

(22)

Basic JUnit usage

continued from prev page

... p u b l i c s t a t i c i n t [] insert ( i n t [] x , i n t n ) { i n t [] y = new i n t [ x . length + 1]; i n t i ; f o r ( i = 0; i < x . length ; i ++) { i f ( n < x [ i ]) break ; y [ i ] = x [ i ]; } y [ i ] = n ; f o r (; i < x . length ; i ++) { y [ i +1] = x [ i ]; } return y ;

(23)

Basic JUnit usage

JUnit can test forexpected return values.

p u b l i c c l a s s E x 1 T es t { @ T e s t p u b l i c void t e s t _ f i n d _ m i n _ 1 () { i n t [] a = {5 , 1 , 7}; i n t res = Ex1 . f i n d _ m i n ( a ) ; assertTrue( res == 1) ; } @ T e s t p u b l i c void t e s t _ i n s e r t _ 1 () { i n t [] x = {2 , 7}; i n t n = 6;

i n t [] res = Ex1 . insert (x , n ) ; i n t [] e x p e c t e d = {2 , 6 , 7};

assertTrue( A r r a y s . e q u a l s ( e x p e c t e d , res ) ) ; }

(24)

Testing for Exceptions

JUnit can test for expected exceptions

@ T e s t (expected= I n d e x O u t O f B o u n d s E x c e p t i o n . c l a s s ) p u b l i c void o u t O f B o u n d s () {

new ArrayList < Object >() . get (1) ; }

expecteddeclares that outOfBounds() should throw an IndexOutOfBoundsException.

If it does

I not throw any exception, or

(25)

Compile and Run JUnit test class

I JUnit plugin in Eclipse IDE:

Right click on your project (or choose when creating project): Build Path → Add Libraries... → Choose JUnit 4.

I Run the tester class as usual in Eclipse. Alt.

I Compile test class:

javac -cp .:pathToJunitJarFile Ex1Test.java

I To run all methods annotated with @Test in a class: java -cp .:pathToJunitJarFile

org.junit.runner.JUnitCore Ex1Test (under Windows: ‘;’ instead of ‘:’)

(26)

Reflection: Extreme Testing

I JUnit designed forExtreme Testingparadigm

I Extreme Testing part ofExtreme Programming (but not depending on that)

(27)

Reflection: Extreme Testing (cont’d)

A few words Extreme Programming

(no introduction here, but see [Myers], Chapter 8)

I Extreme Programming (XP) invented by Beck (co-author of JUnit)

I Most popularagile development process

I Mustcreate tests first, then create code basis

I Must run unit tests forevery incremental code change

I Motivation:

I _{oo programming allows rapid development}

I still, quality is not guaranteed

I aim of XP: create quality programs in short time frames

(28)

Extreme Unit Testing

modules (classes) must have unit tests before coding begins

benefits:

I You gain confidence that code will meet specification.

I You better understand specification and requirements.

I You express end result before you start coding.

I You may implement simple designs and optimise later while reducing the risk of breaking the specification.

(29)

Extreme Testing Example: Class Money

c l a s s Money { p r i v a t e i n t amount ; p r i v a t e C u r r e n c y c u r r e n c y ; p u b l i c Money ( i n t amount , C u r r e n c y c u r r e n c y ) { t h i s . amount = amount ; t h i s . c u r r e n c y = c u r r e n c y ; } p u b l i c Money add( M o n e y m ) {

// NO IMPLEMENTATION YET, WRITE TEST FIRST

} } c l a s s C u r r e n c y { p r i v a t e String name ; p u b l i c C u r r e n c y ( String name ) { t h i s . name = name ; } }

(30)

Write a Test Case for add()

import org . junit .*;

import s t a t i c org . junit . Assert .*; p u b l i c c l a s s M o n e y T e s t {

@Test p u b l i c void simpleAdd() {

C u r r e n c y sek = new C u r r e n c y ( " SEK " ) ; M o n e y m1 = new M o n e y (120 , sek ) ; M o n e y m2 = new M o n e y (160 , sek ) ; M o n e y r e s u l t = m1 . add ( m2 ) ; M o n e y e x p e c t e d = new M o n e y (280 , sek ) ; a s s e r t T r u e ( e x p e c t e d . e q u a l s ( r e s u l t ) ) ; } }

(31)

Example: Class Money

Now, implement the method under test, and make sure it fails

c l a s s Money { p r i v a t e i n t amount ; p r i v a t e C u r r e n c y c u r r e n c y ; . . . . p u b l i c Money add( M o n e y m ) { return n u l l ; } }

(32)

Compile and Run JUnit test class

I JUnit reports failure

(33)

Example: Class Money

First real attempt to implement the method under test

c l a s s Money { p r i v a t e i n t amount ; p r i v a t e C u r r e n c y c u r r e n c y ; p u b l i c Money ( i n t amount , C u r r e n c y c u r r e n c y ) { t h i s . amount = amount ; t h i s . c u r r e n c y = c u r r e n c y ; } p u b l i c Money add( M o n e y m ) {

return new Money ( amount + m . amount , c u r r e n c y ) ; }

(34)

Compile and Run JUnit test class

I JUnit willstillreport failure

I Fix possible defects, until test passes.

I Can you spot it?

(35)

Extend Functionality

Extend Money with Euro-exchange-rate first in test cases

p u b l i c c l a s s M o n e y T e s t {

@Test p u b l i c void simpleAdd() {

C u r r e n c y sek = new C u r r e n c y ( " SEK " ,9.01) ;

M o n e y m1 = new M o n e y (120 , sek ) ; . . . .

}

@Test p u b l i c void addDifferentCurr() {

C u r r e n c y sek = new C u r r e n c y ( " SEK " ,9.01) ;

M o n e y m1 = new M o n e y (120 , sek ) ;

C u r r e n c y nok = new C u r r e n c y ( " NOK " ,7.70) ;

M o n e y m2 = new M o n e y (160 , nok ) ; M o n e y r e s u l t = m1 . add ( m2 ) ; M o n e y e x p e c t e d = new M o n e y (307, sek) ; a s s e r t T r u e ( e x p e c t e d . e q u a l s ( r e s u l t ) ) ; } }

(36)

Common Parts into Test Fixture

p u b l i c c l a s s M o n e y T e s t {

p r i v a t e C u r r e n c y sek;

p r i v a t e Money m1;

@Before p u b l i c void setUp() {

sek = new C u r r e n c y ( " SEK " ,9.01) ; m1 = new M o n e y (120 , sek ) ; } @ T e s t p u b l i c void s i m p l e A d d () { M o n e y m2 = new M o n e y (140 , sek ) ; . . . . } @ T e s t p u b l i c void a d d D i f f e r e n t C u r r () {

(37)

Integrating Test Units

Testing a unit may require:

Stubs to replace called procedures

I Simulate behaviour of component not yet developed.

I E.g. test code that calls a method not yet implemented.

Drivers to replace calling procedures

I Simulate environment from where procedure is called.

(38)

Incremental Testing: Top-Down and Bottom-Up

Explore incremental test strategies, following call hierarchy:

Top-Down Testing

Test main procedure, then go down the call hierarchy

I requires stubs, but no drivers Bottom-Up Testing

Test leaves in call hierarchy, and move up to the root. Procedure is not tested until all ‘children’ have been tested.

(39)

Discussion: Top-down vs Bottom-up Testing

Print Paycheck

Formats and prints paycheck

Staff Database

Person nr:

How would you go about testing the Paycheck system

I Bottom-up?

I Whichdriversdo

you need?

I Top-down?

I Whichstubs do you

(40)

Discussion: Top-down vs Bottom-up Testing

Print Paycheck Formats and prints paycheck Staff Database

Person nr:

I Bottom-up?

start from e.g. Print Paycheck, Staff Database

I Which drivers do you need?

I Driver replacing the caller, here Payroll System, Interface.

I Top-down?

start from e.g. Interface, then Payroll System

I Which stubs do you need?

I Stubs replacing called

(41)

Top-Down Testing: Pros and Cons

Advantages of Top-Down Testing

I Advantageous if major flaws occur toward top level.

I Early skeletal program allows demonstrations and boosts morale. Disadvantages of Top-Down Testing

I Stubs must be produced (often more complicated than anticipated).

I Judgement of test results more difficult.

(42)

Bottom-Up Testing: Pros and Cons

Advantages of Bottom-Up Testing

I Advantageous if major flaws occur toward bottom level.

I Judgement of test results is easier. Disadvantages of Bottom-Up Testing

I Driver units must be produced.