Unit testing with examples in C++

Unit testing with examples in C++

Institute  for  Cyber-­‐Enabled  Research  

Agenda

• Test-driven development

• Unit testing and frameworks

How this workshop works

• We are going to cover some basics.

– A few of hands on examples

• Exercises are denoted by the following icon in this

Test-driven development (TDD)

• TDD is a software development process which relies

on the repetition of a very short development cycle

1. Add a failing test case that defines a desired improvement or new function

2. Run all tests and see if the new one fails 3. Write some code to pass the test

4. Run tests

Test-driven development (TDD)

• Let’s assume you want to develop a software

module which calculates

– defInt(f(x), a, b)

• You need to know if your new module works

correctly.

• So, before you start coding, a test case is prepared.

• eg: defInt(f(x), a, b) == 1/2

TDD workflow

Software testing

• Many different forms of tests:

– unit tests:

– integration tests: to test if Individual units are combined and functioned as a group.

– regression tests: to uncover new software bugs in existing functional

– performance tests: to determine how a system performs in terms of responsiveness and stability under a particular

Unit testing

• Unit testing is a method by which individual units of

source code are tested to determine if they are correctly working.

• A unit is the smallest testable part of an application.

Benefits of unit testing

• Facilitate changes: unit tests allow programmers to

refactor code at later date, and be sure that code still works correctly;

• Simplify integration: unit tests may reduce

uncertainty in the units, and can be used in a bottom-up testing style approach.

• Living documentation for the system: easy to gain a

How to organize tests

• Each test case should test only one thing

• A test case should be short

• Test should run fast, so it will possible to run it very

often

• Each test should work independent of other test

• Tests should NOT be dependent on the order of their

• How should a test program report errors? Displaying

an error message would be one possible way.

– requires inspection of the program’s output after

each run to determine if an error exists.

– Human inspection of output message is time

consuming and unreliable.

• Unit testing frameworks are designed to automate

those tasks.

Test methods: man in the loop

if( something_bad_detected )

Test methods: EXIT_SUCCESS/EXIT_FAILURE

• There are several issues with the test above

– You need to convert is_valid result in proper result code – If exception happens in test_object when is_valid

invocation, the program will crash.

– You won’t see any output, if you run it manually.

#include <my_class.hpp> int main( int, char* [] )

{

my_class test_object( "qwerty" );

return test_object.is_valid() ? EXIT_SUCCESS : EXIT_FAILURE; }

• The Unit Test Framework solves all these issues. To

integrate it the above program needs to be changed to:

Test methods: Unit testing frameworks

#include <my_class.hpp>

#define BOOST_TEST_MODULE MyTest #include <boost/test/unit_test.hpp> BOOST_AUTO_TEST_CASE( my_test ) {

my_class test_object( "qwerty" );

BOOST_CHECK( test_object.is_valid() ); }

• To simplify development of unit tests, unit test

frameworks are usually used.

• Almost any programming language has several unit

testing frameworks.

Unit testing frameworks in C++

• There are many unit testing frameworks for C++

– Boost.Test, Google C++ Testing Framework, etc, etc, etc!

• Boost.Test

– Suitable for novice and advanced users

– Allows organization of test cases into test suites

– Test cases can be registered automatically and/or manually

– Fixtures (initialization and cleanup of resources) – Large number of assertion/checkers

• exceptions, equal, not equal, greater, less, floating point numbers comparison etc.

Preparation

• connect to the MSU hpc

– open a terminal and do

1. ssh [email protected] 2. ssh dev-intel10

3. module load powertools 4. getexample boostUnitTests 5. cd boostUnitTests

simpleUnitTest1.cpp

#define BOOST_TEST_MODULE Simple testcase // name of this test #include <boost/test/unit_test.hpp> //header file for boost.test // a module that we want to develop

int addTwoNumbers(int i, int j ) { return 0; } //Here is a test BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(addTwoNumbers(2, 3) == 5); }

simpleUnitTest1

• Now let’s build and run it

• You should be ~/boostUnitTests. Please do

1. mkdir build 2. cd build 3. cmake ../ 4. make

5. ./simpleUnitTest1

• It will give you an error message because we did not

simpleUnitTest1

• A failed BOOST_CHECK will not exit the test case

immediately - the problem is recorded and the case continues. To immediately fail a test, BOOST_REQUIRE is used.

#define BOOST_TEST_MODULE Simple testcases #include <boost/test/unit_test.hpp>

int add(int i, int j ) {return 0;}

BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(add(2, 3) == 5); }

simpleUnitTest2.cpp

#define BOOST_TEST_MODULE Simple testcase //declares name of this test #include <boost/test/unit_test.hpp> // header file for boost.test

// a module that we want to develop int addTwoNumbers(int i, int j ) {

return i - j; // need to be fixed to pass the test

}

//Here is a test

BOOST_AUTO_TEST_CASE(addition) {

BOOST_CHECK(addTwoNumbers(2, 3) == 5);

BOOST_CHECK_MESSAGE(addTwoNumbers(2, 3)== 5, "<addTwoNumbers> has some problem!");

BOOST_CHECK_EQUAL(addTwoNumbers(2, 3), 5); BOOST_REQUIRE(addTwoNumbers(2, 3) == 5);

BOOST_WARN(addTwoNumbers(2, 3) == 5); //will never reach this check }

simpleUnitTest2

• Run and verify the difference between assertions.

Assertions

• BOOST_WARN is not checked here, because the test

has failed with BOOST_CHECK_REQUIRE

BOOST_AUTO_TEST_CASE(addition) {

BOOST_CHECK(addTwoNumbers(2, 3) == 5);

BOOST_CHECK_MESSAGE(addTwoNumbers(2, 3)== 5, "<addTwoNumbers> has some problem!");

BOOST_CHECK_EQUAL(addTwoNumbers(2, 3), 5); BOOST_REQUIRE(addTwoNumbers(2, 3) == 5);

BOOST_WARN(addTwoNumbers(2, 3) == 5); //will never reach this check

Testing tools/Checkers

• WARN

– produces warning message check failed, but error counter does not increase and test case continues

• CHECK

– reports error and increases error counter when check fails, but test case continues

• REQUIRE

Testing tools/Checkers

• expression examples

– BOOST_WARN( sizeof(int) ==sizeof(short)); – BOOST_CHECK(i == 1);

– BOOST_REQUIRE (i > 5); – BOOST_REQUIRE(‘h’, s[0]);

• If the check fails, Boost.Test will report the line in the source code where this occurred and what condition was specified.

More Assertions

• CHECK, REQUIRE, and WARN (with “—log_level=all”)

• BOOST_CHECK_MESSAGE

• BOOST_CHECK_EQUAL

• The full list of available assertions can be found

simpleUnitTest3.cpp

• If you have many test

cases in one program, then their maintenance could be hard.

– test suite is available

– In a normal run, you won’t see that the tests have been categorized.

– With “—log_level=test_suite” or “—log_level=all”, you will.

#define BOOST_TEST_MODULE Suites #include <boost/test/unit_test.hpp>

int add(int i, int j) { return i + j;} BOOST_AUTO_TEST_SUITE(Maths) BOOST_AUTO_TEST_CASE(addition) { BOOST_CHECK(add(2, 2) == 4); } BOOST_AUTO_TEST_SUITE_END() BOOST_AUTO_TEST_SUITE(Physics) BOOST_AUTO_TEST_CASE(specialRelativity) { int e = 32; int m = 2; int c = 4; BOOST_CHECK(e == m * c * c); } BOOST_AUTO_TEST_SUITE_END()

Fixtures

• Fixtures are special objects that are used to

implement setup and cleanup of data/resources required to execution of unit tests.

• Separation of code between fixtures and actual test

code, simplifies the unit test code, and allows the same initialization code to be used for different test cases and test suites.

Fixtures example

and you can use it in the following way:

struct MyFixture { int* i;

MyFixture() i = new int; *i = 0;} ~MyFixture() {delete[] i;}

};

BOOST_AUTO_TEST_CASE (test_case1) {

MyFixture f;

// do something with f.i }

Boost.Test’s fixture macro

and you can use it following way:

#define BOOST_TEST_MODULE fixture example #include <boost/test/unit_test.hpp> struct F { int* i; F(): i(1) {} ~F() {} }; BOOST_FIXTURE_TEST_CASE(simple_test, F) { BOOST_CHECK_EQUAL*i, 1); }

simpleUnitTest4.cpp

• The “Massive” fixture was

created, it holds one

variable m, and it is directly

accessible in our test case.

#define BOOST_TEST_MODULE Fixtures #include <boost/test/unit_test.hpp> struct Massive { int m; Massive() : m(3) { BOOST_TEST_MESSAGE("setup mass"); } ~Massive() { BOOST_TEST_MESSAGE("teardown mass"); } }; BOOST_FIXTURE_TEST_SUITE(Physics, Massive) BOOST_AUTO_TEST_CASE(specialTheory) { int e = 48; int c = 4; BOOST_CHECK(e == m * c * c); } BOOST_AUTO_TEST_CASE(newton2) { int f = 10; int a = 5; BOOST_CHECK(f == m * a); } BOOST_AUTO_TEST_SUITE_END()