Contents
• Beyond testing
• Testing levels
• Testing techniques
• TDD = fail early
• Automate testing = fail often
• Tools for testing
Testing in different levels
§ Unit testing
§ Does a single object work? § Integration testing
§ Do multiple objects work together? § Functional testing
§ Does my application work?
§ Acceptance testing
§ Does the customer like my application? § Regression testing
§ Does a bug fix result in another fault in the application?
Techniques used in testing:
§ white-box testing
§ You know the code and you do your best to break the code
§ grey-box testing
§ You peek a little inside, e.g. know the architecture § black box testing
§ Focuses on input and output
Fail early i.e. Test-Driven Development practice
W rite a Test Case --> W atch it Fail --> Fix it --> W atch it pass --> Refactor the code
Red
Green Refactor
Write a test to fail
Make the code work Tidy up, eliminate
§ All failing § Order of
Pros and cons of TDD
§ It’s hard to learn and master
§ You have to accept that some of the tests become obsolete (you have to
delete them)
§ You trust too much on your tests (remember: they are code just like any other code)
§ You might try to fix the tests to show green by making ad hoc decisions forgetting to solve the real problem
§ W riting tests first require you to really consider what you want from the code § Creates a detailed specification (you
understand the problem better)
§ Less time spent in the debugger and when it is required you usually get closer to problem quickly
§ Tells you whether your last change (or refactoring) has broken previously
working code
§ Allows the design to evolve and adapt to your changing understanding of the problem.
§ Unit tests are simple and act as documentation for the code
Fail often i.e. test automation
§ Automate the tests, and the tests can be run whenever
§ Automate the tests, and free developers time to more challenging problems
§ Automate the tests, and run them on a separate server not in the developer’s setup
§ Automate the tests, and you get a regression test suite
Continuous integration
§ Continuous integration wraps version control, compilation and testing into a single repeatable process
§ CI is a software development practice where members of a team integrate their work
frequently, usually each person integrates at least daily - leading to multiple integrations per day.
§ Each integration is verified by an automated
build (including test) to detect integration errors as quickly as possible.
Continuous integration server like Jenkins
Continuous integration server like Jenkins
§ Shows how tests are succeeded in
separate builds
§ Shows for example FindBugs static
analyzer trend
Dm: Found reliance on default encoding in sofa.model.MapBuilder.loadMap():
Found a call to a method which will perform a byte to String (or String to byte) conversion, and will assume that the default platform encoding is suitable.
Tools for unit testing
§ There are hundreds of tools for different levels of testing
§ One popular family is <x>Unit family for different programming languages: Junit, Cunit, CppUnit, PHPUnit, PyUnit, JsUnit,….
§ This family can be used for unit testing but also for integration testing/ functional testing
Unit Test Types
§ Basic -- cases with small to medium sized inputs that are so simple they should obviously work. § Calling every method in several ways
§ Call assertEquals(x, y) also in case it returns false
§ Edge -- these are also cases that are simple but represent edge conditions -- the empty string, the empty list, etc.
§ Advanced -- harder, more complex cases. Testing e.g. using the knowledge about the code, synchronization, complex algorithms,
<x>Unit
§ All of the <x>Unit family of testing libraries rely on test cases where you basically make a
proposition or claim of the behaviour of the program.
§ You claim that if your programs gets a certain
input, it will give back an output you want. These are called assertions.
§ Junit is introduced in the Java testing session
Assertions (many more available)
§ fail(String message)
§ Gives a message that test didn’t go through § assertTrue(String message, boolean condition)
§ Checks if the condition is true
§ assertFalse(String message, boolean condition) § Checks if the condition is
§ assertEquals(Object expected, Object actual) § Checks if the parameters expected and actual
C and C++
§ CodeLite and Code::Blocks IDE’s are introduced in the C testing session
§ CodeLite has integrated UnitTest++ and this will also be introduced in C testing session
§ CodeLite is not <x>Unit family but has something similar:
CHECK_EQUAL(10, i);
Acceptance testing
§ Acceptance tests are black box test cases that are jointly written by a developer and a
customer.
§ An acceptance test is a concrete situation, or scenario, that the system might encounter when using the functionality of a user story.
Summary
§ Change your perspective from developer to tester!
§ Write tests as early as possible!
Further reading
§ Head First Software Development, By: Dan
Pilone & Russ Miles Publisher: O'Reilly Media, Inc.
§ http://www.agiledata.org/essays/tdd.html by
Scott W. Ambler
§ The Art of Agile Development: Test-Driven Development by James Shore
http://www.jamesshore.com/Agile-Book/test_driven_development.html § And many many more….
