In contrast to the process described above, the motivation for testing with test- driven development is not the verification of the test objective, but its implementation (explicit) and its design (implicit). The tests are therefore created and executed before the implementation. Details have already been outlined in the description of the development cycle of TDD in chapter 1.3.2.
Test-driven development can be used with all testing techniques, but two types are most commonly applied: acceptance testing and unit testing [172, 173].
In the acceptance test the expected behaviour of the developed system is described from the customer’s point of view. The requirements or, in general, the use cases are translated to an executable form. The time horizon of acceptance tests is a mid to long-term period and covers at least one iteration of the development process. In fact, the aim of an iteration can be defined as the completion of a set of acceptance tests. For such tests all three software levels and all four testing environments (cf. Chapter 2.3.1) are sufficient, but typically the highest level is chosen to test the system in its real environment. Based on this high-level test, a number of subtests can be created which consider the characteristics and features of the respective level and environment (top-down approach).
The following rules describe the role of acceptance tests within the development cycle of TDD:
Rule 1 : All requirements of a system have to be covered by acceptance tests.
Rule 2 : A new requirement is only allowed to be implemented if the respective acceptance test fails.
In contrast, the goal of the unit tests is the realisation of the requirements based on the smallest entity, the software unit. The idea behind this is that every change must follow a failing test, no matter how small the change is. In other words, the rules for unit tests are:
Rule 3 : The functionality of a unit has to be covered by unit tests.
Rule 4 : A unit is only allowed to be modified if the respective unit test fails.
Rule 4 requires implicitly that if a defect occurs which is not detected by an unit test, e.g. if it is reported by the customer, then this defect has to be first reproduced by a test before the unit is modified.
The unit tests can be designed using the techniques of black-box testing as well as white-box testing providing the variety of both approaches. Furthermore a clas- sification into two types of unit tests is proposed [84]. The first group analyses and specifies the behaviour of the system (“what it should do”), while the second group helps to implement the controller in a systematic and traceable way (“how it is done”). In addition, a third group is necessary for the verification of the unit and its implementation process [174].
The development of the units is done with a bottom-up approach, i.e. first a single unit is tested and realised, then a number of units can be integrated for further development. However, the testing of two or more units together can also start before the unit is completely implemented, e.g. to develop the interaction between the units. The interrelationship between acceptance and unit tests is shown in Figure 2.15. The activities “write an acceptance test” and “run the test(s)” can have several mean- ings in the context of model-based development depending on the test environment:
Write a unit test. Run the test(s). Make a change. Refactor [Fail] [Pass] Write an acceptance test. Run the test(s). [Pass] [Fail] Error
Figure 2.15: Acceptance and unit tests within the development cycle of test-driven development
• With MiL and SiL environments, an acceptance test consists of a plant model, its initial values and the (time-dependent) definition of additional stimuli, e.g. disturbance values. The test is executed by simulating this model together with the test objective.
• The definition of HiL tests is governed by the simulation environment of the test bench and usually achieved by the specification of test vectors or scripts which control this environment. Thus the variables for the models of the environment, e.g. the vehicle’s environment with the road’s course or other vehicles and the driver, are configured based on events, e.g. the driver will hit the brake pedal when the distance from the vehicle ahead is less than 20 meters. Then the tests are run automatically by the test bench.
• A target test is typically described through the manual execution of the system by its user, e.g. a typical acceptance test for a vehicle system is a (driving) ma-
noeuvre [175, 176]. This execution can be partly or fully automated depending on the kind of the system. Continuing the example, a partial automation can be achieved by steering or braking robots as well as sled tests (crash tests). Other methods include so-called endurance or stress tests, during which the system is either run for a longer period or stimulated with a high workload.
In contrast, the activities of unit testing are always carried out at the model level, i.e. a test is written by building a model and run by simulating both the model and the test objective.
The evaluation of the tests is achieved by the methods described above, see Chapter 2.3.3. Here, the use of assertions is not limited to the automated tests with MiL, SiL and HiL, but can be also used with target testing. The driver compares the system’s reaction by means of measured data with the expected results after he has performed a manoeuvre, and marks the test with passed or failed.8
Finally the developer is allowed to “make a change” in terms of model-based de- velopment if at least one unit test fails. The unit is implemented by creating or modifying a model exactly in such a way that the test is passed.
Another important activity of test-driven development is refactoring. Refactor- ing specifies the improvement of the system’s structure while retaining its present behaviour [177, 178, 179]. Thus, this step relies on the results of the tests as they provide an executable form of the system’s specification and requirements. The meth- ods of refactoring include the improvement of readability and comprehensibility as well as the iterative refinement and optimization of architecture and design. More- over, the developer draws conclusions from reviews of the implemented models and from the unit tests, e.g. the duplication of code, to modify the system in terms of modularity, interfaces, maintainability, and testability.