Related Work on Model Based Testing

Model Based Testing has led many researchers to use UML diagrams such as state-chart dia- grams, use-case diagrams, sequence diagrams, etc. to generate test cases. The major advantage of these model-based testing techniques is increased productivity and quality by shifting the testing activities to an earlier stage of the software development process and generating test cases that are independent of any particular implementation of the design.

Approaches to the generation of representative scenarios for testing often use intermediate rep- resentations for the activity diagrams to be tested, such as (1) normalized activity diagrams (reduced to certain constructs)[54], (2) special directed graphs [66, 58], (3) Petri-nets [65] or (4) test trees [53, 67, 68].

UML Based Test Generation

Briand et al. [71] investigated a UML-based approach to generate tests from system models. The aim of the approach is to generate functional tests, oracles and so-called test drivers from refined system models, whereby the artefact test driver refers to test execution. Specifically, Briand et al. use activity diagrams with descriptions, sequence diagrams, collaboration diagrams and class diagrams to generate test cases. However, the use of activity diagrams in the authors’ approach serves only the purpose to derive superordinate system functions that have to be covered by tests. The activity diagrams only illustrate system functions on a high level of abstraction, but do not provide information on the permitted execution sequences of individual actions. For this reason, the authors also analyse the business logic of the application to determine permitted function sequences based on UML activity diagrams that are assigned to individual user roles in the system. In addition to the use case diagram, domain experts expertise must be consulted. In the approach examined by Briand et al., test data is defined by annotating individual actions in the activity diagrams with informal data objects. The method studied in this dissertation examines the use of elements from meta models for test case modelling. In contrast to their approach in this work the model is supplemented with test data and context parameters through refinement steps.

Cavarra et al. [72] investigated an approach to automatically generate test cases from UML models. In the studied concept, the authors transform UML models into models that represent test cases and test data. To integrate test-relevant data into UML models, the authors design a UML profile that enhances the meta model of UML. The authors base their approach on the assumption that software quickly achieves complexity that can no longer be fully grasped by human actors. Modelling should help to abstract from the technical details and reduce complexity. Cavarra et al. further argue that complexity reduction is necessary, especially in the field of testing, since the state space of possible combinations of inputs into a software system is rapidly increasing to an unmanageable extent. Furthermore, the authors argue that in principle any modelling language could be used for the test generation process. However, the

decisive criterion for acceptance by the user is that no additional effort is required to learn the modelling language. Since UML is a standard in software engineering, Cavarra et al. use it as the basis for their research. However, in contrast to the approach examined in this thesis, state chart diagrams are used by the author to represent the test model.

In 2003, Robinson [73] addressed the challenges and opportunities of model-based test genera- tion. The author bases his argumentation for generating tests from models instead of manual creation on the observation that testers are often less technically oriented than software devel- opers and are involved in the software process relatively late. A lack of technical involvement in the software to be tested and a low relation to the project history reduce the quality of tests. A model-based approach can also help less technically experienced testers to quickly develop an abstract understanding of the SUT and focus test development on key product features. The basic idea here is to understand tests no longer as artifacts developed parallel to the software product, but as executable specifications. The author argues that natural language specifica- tion documents are often too imprecise for effective test case creation to ensure adequate test coverage. Robinson also discusses the benefits of model-based test generation in terms of the number of individual test cases that are used as a metric of productivity from a management perspective. For complex software, a large number of tests is quickly created. Changing the specification of the software product may render some or all of these tests obsolete and may require the creation of new tests. Manual creation may require a lot of work to revise existing tests, while the effort to generate tests from a model is reduced to modifying the model. UML activity diagrams basically describe behaviours, especially processes and procedures. Cor- responding descriptions are therefore particularly relevant for testing, since they can serve as a test model on the one hand and as a test object on the other hand. In principle, many papers suggest that activity diagrams are primarily suitable for model-based testing [52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63].

Some of the mentioned approaches differ considerably, e.g. with regard to the supported language features or the UML version. There are also works for direct testing of the behaviour described by the diagrams themselves [64, 65]. Most approaches covered here are very unlikely to be generalised as they are often based on a very specific scenario and thus are not applicable to other situations. However, general ideas or techniques can be used.

Algorithms for test case generation

Not only different intermediate representations are used to represent the test model. Also different algorithms are used to traverse the model in order to extract test cases. Technically, test generation and optimization methods are usually based on:

1. genetic algorithms [69], 2. optimization heuristics, as

b) adaptive agents [68], 3. search algorithms [66], 4. model-checking [64, 65],

5. random test generation [70, 55]

The approach chosen in this work also follows the general pattern of model-based testing tech- niques as defined in Section 2.2.2. For the derivation of test cases, a search algorithm has been chosen. In contrast to genetic algorithms, which can lead to different test cases, a breath first search was used to ensure a deterministic behaviour during test case generation. Additionally, one draw-back of genetic algorithms is that they usually do not scale well with complexity, which can be very challenging in larger activity diagrams. However, all the afore mentioned algorithms can also be used, provided that the specific extensions are preserved in terms of the test execution framework (in particular the virtual RFID environment).