In the following we describe the test tasks in more detail and how dependencies effect the ease to perform them.
2.2.1
Define Test Order
A good test order helps to minimize the required number of stubs and drivers during unit and integration testing. The actual test order depends on the test strategy (e.g. bottom-up or top- down) and the class dependencies existing within the system. Effect of dependencies
If the classes and dependencies form a tree structure, it is trivial to identify a meaningful test order1, and the degree of freedom in choosing the test order is large.
Adding additional dependencies reduces the freedom in choos- ing the test order. Additional dependencies which introduce dependency cycles make it more difficult to identify a meaningful test order [Kung94b] [Kung94c] [Over94] [Wint99] and may require the implementation of stubs (see Section 2.2.3).
Note: Adding dependencies also has an impact on the overall test process because it reduces the opportunities for parallel testing.
2.2.2
Define Test Cases
Each test case has to describe the initial set of the classes involved, the initial state of the objects, the input parameters, global variables, and required method calls (or user interactions) as well as the expected test results like GUI output, return val- ues, object states, and exceptions thrown.
For a system of reasonable size it is practically impossible to cover all possible input combinations with help of test cases. Therefore, different test coverage criteria have been developed to help in finding a small but sufficiently large set of test cases with a high probability to uncover most of the errors.
1 In this case, the test order starts with leaf nodes and continues with nodes upwards the tree where all supplier nodes have already been tested.
18 Chapter 2: Dependencies and Test Tasks
Effect of dependencies
Adding a new dependency between a client class (the class under test) and a supplier class may for example a) increase the number of the client class’s methods and/or method parameters or b) make the control flow of the client class dependent on the state of the supplier class, which likely increases the required number of black-box test cases.
If the new dependency manifests as a change within a method body of the client class, additional white-box test cases may be required as well, especially if the state of the supplier class is rel- evant for the control flow of the class under test.
CUT = class under test Note: In the remainder, we will use the abbreviation CUT for the class under test.
2.2.3
Implement Stubs, Mocks, and Drivers
Stubs, mocks [Link01], and drivers are simple substitutes for classes (or interfaces):
• Stubs allow to test a class in isolation or to substitute classes which haven’t been tested yet.
• Mock objects additionally improve the ability to observe (intermediate) test results1.
• Drivers are used to control the CUT directly.
The need to implement stubs, mocks, and drivers depends e.g. on the integration strategy chosen.
Effect of dependencies
Introducing new dependencies increases the number of stubs required to test classes in isolation. Dependencies introducing dependency cycles require the implementation of stubs if the strategy is to break the cycles during testing.
2.2.4
Build System
Before the first test run, all executables of the classes involved in the test have to be built from the source code. If the CUT or its supplier classes have been changed before re-running a test, existing executables have to be rebuilt as well.
1 Mock objects may report intermediate test results in terms of the (se- quence of the) method calls to the mock object and the corresponding values of parameter objects.
Chapter 2: Dependencies and Test Tasks 19
Effect of dependencies
Adding dependencies means that more classes have to be re- compiled on average after program changes (see [Cold96]) which may increase the duration of the compile-test-debug cycle significantly.
2.2.5
Instantiate and Initialize Objects Involved in Tests
Objects of the classes involved in a test have to be instantiated and often initialized during test setup. Initialization may include the establishment of appropriate object links.
Effect of dependencies
An additional dependency to a supplier class often increases the number of objects that have to be instantiated and initialized. For example, if an instance of the supplier class has to be provided as a parameter object to the method under test, the test code has to create this supplier instance. If the CUT creates the instance of the supplier class itself, the test code is freed from this responsibility, but such direct creation of supplier instances can cause other test problems (see Section 7.2.1 and Section 7.3.1).
2.2.6
Execute Tests
Test case execution means to call a method of a class or to invoke user operation(s) provided by a graphical user interface (GUI). This can be done manually or automated.
Effect of dependencies
Adding a dependency from a CUT to a supplier class which involves time consuming operations like user input, access to a database, or access to the Internet increases the overall time needed for test execution.
2.2.7
Observe and Analyze Test Results
During and after test execution the intermediate and final results need to be observed. Test results include values of return parameters, the (hidden) state of objects, exceptions that have been thrown, and other kinds of system output.
20 Chapter 2: Dependencies and Test Tasks
The test results observed have to be compared with expected test results. In case of failures it is necessary to describe them with as much details as necessary to facilitate isolation of errors. Effect of dependencies
The more classes a CUT depends on the more instances will be involved in test execution and the more intermediate and final object states have to be observed as part of the overall test result (in general) which increases the related effort.
2.2.8
Isolate Errors
In case of a wrong test result it is necessary to track the failure down to the faulty class(es) and code statement(s): analytically based on supplemented information (like system traces) or empirically by adding, removing, exchanging or additionally test- ing objects of other classes involved.
Effect of dependencies
A higher number of dependencies makes fault isolation more dif- ficult because an error may hide in a larger number of classes [Cold96]. Cyclic dependencies pose a special problem because they lead to re-entrance situations [Szyp97] which make it harder to understand the code and to isolate faults.
2.2.9
Identify Change Impact
During maintenance, test cases have to be rerun in order to make sure, that no new errors have been introduced while fixing errors or adding new functionality. The number of test cases which have to be rerun during the regression tests can be reduced by identifying the classes which are actually effected by the implementation change.
Effect of dependencies
A higher number of dependencies causes more effort to deter- mine the set of classes effected by the change.
2.2.10
Maintain Test Cases
Test cases related to a given class may need to be updated if this class or one of its supplier classes changes.
Chapter 2: Dependencies and Test Tasks 21
Effect of dependencies
Adding dependencies to a system reduces the independence of unit test cases and increases the number of test cases which have to be maintained after an implementation change.
2.2.11
Automate Test Cases
Test automation means that some or all steps necessary during test execution, test result observation, and analysis are auto- mated. This requires that test results are presented in a way which facilitates comparison with expected results without the need for human intervention.
Effect of dependencies
Adding a dependency may have a negative effect on testing if the supplier class causes problems for test automation.