Model Integration Level Test

Analyzing the integration level test the cross-cutting nature of functionality and failures occur- rence is considered. This enables quality assurance (QA) to be approached in terms of interact- ing services, which is a potential candidate for dealing with the integration problems [EKM+07]. Service-orientation means, in this context, a design paradigm that specifies the crea- tion of automation logic in the form of services. A service itself is a cross-cutting functionality that must be provided by the system. Like other design paradigms, it provides a means of achieving a separation of concerns [All06]. The services are sometimes also referred to as sys- tem functions or functionalities in the automotive domain. Services identify partial behaviors of the system in terms of interaction patterns [Krü05, EKM06, BKM07]. Based on the interactions between services the test specifications are developed.

5.6.1 Test Specification Design Applying the Interaction Models

The analysis of the system starts at the requirements level, where features and functions of new software within a car [KHJ07] are described. The requirements are used as a basis for interac- tion models. These models present the behavior between different software parts. In the follow- ing the extended Message Sequence Charts (MSCs) [ITU99, Krü00] combined with the hybrid Sequence Charts (hySC) [GKS99], called hySC for testing (hySCt), are applied as a modeling technique. With this practice, both discrete and continuous signals exchanged between the components, can be expressed. Such hySC can be advantageously used in the early phases of the development process, especially, in the requirements capture phase. The interacting compo- nents are specified as the roles and the functional relations between them are the services. In the upcoming paragraphs the syntax and some semantics extensions of [Krü00] (as compared to traditional MSCs [ITU96, ITU99] combined with the semantics given by [GKS99] are ap- plied. In particular, (1) arrows to denote events are used. Then, (2) angular box denotes condi- tions on the component’s variables suggested by [GKS99]. A pair consisting of an arrow and an angular box defines (3) the conditions (i.e., states defined by a set of signal feature (SigF)) be- ing mainly a result of sending the information continuously. This enables the notation for con- tinuous behavior to be clarified. However, the semantics remains the same as [GKS99] defined: (4) a continuous global clock exists and an abstract time axis is available for each component. The components occurring in the sequence charts are connected by channels along which in- formation exchange occurs. It is possible that more than one signal (i.e., message) appears si- multaneously (5). The dashed vertical lines are used to denote simultaneous signals appearance. A simple example is given in Figure 5.28. Also, (6) triggers, (7) parallel signal receiving, (8) an

alternative, and (9) local timers are available. These are specified in [ITU99]. It is left open

The resulting hySCts are translated to the MiLEST syntax and semantics. Hence, the hySCts are transformed to the VFs and interpreted in terms of their execution.

Role 2

Role 1 informs Role 2

about its state

message 1 Role 1 message 2 Role 3 state 1 state 2 message 3

Figure 5.28: Basic hySCt.

The hySCts are the interaction models representing the services as shown in Figure 5.28. A ser- vice is defined by the interaction of its roles. In an interaction, a role represents the participa- tion of a system entity in an interaction pattern. Between roles, different information can be exchanged. These are: a message, an event, a continuous signal, or a reference to another ser- vice. Compound interaction elements consist of a number of interaction elements composed by operators. Instances of operators are sequence, loop, alternative, parallel, and join, representing sequential composition, repetition, choice, parallel, and join composition, respectively [EKM06]. The state in an angular box indicates a condition and remains unchanged until a new state appears. Please note that this interaction model abstracts from concrete notations – in the context of this work a combination of selected modeling dialects has been used.

HySCts combined with the SigF paradigm introduced in Chapter 4 of this thesis enable the flow

of information (i.e., signals, events, messages) between different components (i.e., roles) to be designed without specifying how this flow should be realized and how the components process the inputs to produce valid outputs. This abstract design method is powerful enough for specify- ing the integration level test. Further on, the resulting elements are incorporated into and exe- cuted together with the SUT model.

Role 2 message 1 Preconditions Assertions Role 1 message 2 Role 3 state 1

Figure 5.29: HySCt including VFs Concepts.

In Figure 5.29, a simple example of a hySCt enriched with additional elements is given. In ref- erence to the test specification, preconditions and assertions boxes being the constituents of a VF are added at the top of the diagram so as to identify which information should be trans- formed to the appropriate VF part. The dashed lines indicate the simultaneity.

Then, it is aimed to facilitate the retrieval of the running VFs from the hySCts. Hence, the boxes called preconditions and assertions present at the hySCts diagrams are reused. They recall the notion of the test specification. The role of each such VF, in this context, is to detect the SUT failures. Every hySCt may result in one or more VFs depending on the complexity of the re- quirement on which it was built and its own complexity.

5.6.2 Test Data Retrieval

At this point such a test specification can be utilized twofold: (1) as an information source for the test data generator for further test design refinement; (2) to evaluate the SUT behavior dur- ing the test execution (being the primary aim of a VF existence).

Since (2) has already been discussed in the previous sections of this thesis (cf. Section 4.4), herewith more attention will be given to (1).

Applying the algorithms for test stimuli derivation as described in Sections 5.3 – 5.5 the redun- dant test cases would be obtained. The redundancy (i.e., in the sense of duplication) is caused by generating similar test data for different VFs because of the existence of similar precondition sets. Additionally, it may happen that several VFs include exactly the same sets of pre- preconditions, resulting from the hySCts, next to their original preconditions. This situation oc- curs especially at the system integration level test when the previous test scenarios play a sig- nificant role for the current test scenario. In that case, the test objective being checked at the moment may only be reached when the SUT is brought to a predefined state.

Hence, as a solution to this problem, not only the VF’s preconditions, but also higher-level

hySCts (HhySCts) decide on the contents of generated test data sets. HhySCts are defined simi-

larly as high-level MSCs in [ITU99, Krü00]. They indicate sequences of, alternatives between, and repetitions of services in two-dimensional graphs – the nodes of the graph are references to

lated into basic hySCts without loss of information. Generally, the rule applies that only those VFs, where the entire sequence of interactions within one single service is caught, should be considered for the test data generation. If this is the case the number of the test data sets does not match the number of VFs. In fact, the number of test data sets is less than the number of VFs and it matches the number of services present in the higher-level hySCt. This is an advan- tageous situation in terms of testing as all possible behavioral sequences are still covered and the tests are not redundant.

Additionally, the sequences of the generated sets of the test data are obtained from the higher- level hySCts by analyzing the paths of those charts are analyzed. The algorithm for obtaining possibly lots of the meaningful sequences results from the white-box test criterion commonly known as path testing [ISTQB06].

Further on, when the test data sets are set and their sequencing algorithms have been defined, the automatic test data variants generation and their combination methods may be applied with- out any changes as described in Sections 5.4 – 5.5.

5.6.3 Test Sequence versus Test Control

In this thesis, at the component level test, the single test case consists of several test steps. The test control allows for specification of the execution order for such test cases including their different variants. Hence, no additional concept for a test sequence is used.

At the integration level test the situation changes. Here the test sequence is derived from the

HhySCts. It includes a set of test cases that need to be executed in the order specified by this

sequence. Then, the test control enables to manage all the resulting test sequences and their variants. An example of such circumstances will be given in Section 6.4.