Testing with Task Sequences

Task sequences represent interaction scenarios between system and environ- ment. The test selection criteria that where introduced in the last section enable the generation of task sequences that cover feature interaction scenar- ios where faults are likely to occur. Hence, each task sequence represents a scenario that must be tested in order to ensure that no feature interaction results in a feature interference.

A task sequence describes a sequence of task executions and the correspond- ing task modes. Figure 6.1 shows a simple task model that describes the tasks Incoming Call and Listen to Radio. One interesting task sequence is the one where the incoming call suspends the radio. Figure 6.2 shows the correspond- ing task sequence.

Listen to Radio* Select

Station <<enable>> Incoming Call <<suspend>>

Fig. 6.1. Task model.

A tester can now perform this task sequence and test the system for correct behavior by executing the following steps:

1. Start the radio by selecting a station. 2. Start an incoming call.

3. Test if the incoming call is signaled. 4. Test if the radio is suspended.

The tester executes a task sequence by implicitly mapping the tasks to the corresponding environment inputs and the task modes to the corresponding system states. For example, the task Select Station is mapped to the IDrive controller inputs which are necessary to select a station and the task mode suspended from the task Listen to Radio is mapped to testing whether the radio is suspended. The tester performs this mapping based on the knowledge of the system and the system specication.

However, the eort of manual test execution is high, in particular when test cases must be repeated multiple times due to dierent system congurations or due to system changes. Furthermore, test case generation potentially produces a large number of test cases. For example, the interleaving coverage criterion produces a task sequence for each interleaved execution of two independent tasks. Thus the automated execution of test cases is desirable in order to reduce testing eort and to enable a continuous testing process. This requires executable test scripts that automatically trigger and observe the system. Therefore it would be desirable for a test script to automatically perform the steps of a manual tester. Listing 6.1 shows such a Python test script for the

Environment SelectStation ListenToRadio IncomingCall Start Start Active Active Stop Stopped Start Active Suspended Suspend

...

6.2 Related Work 75 task sequence in Figure 6.2. The test script can be executed automatically and perform the same tests as the manual tester.

# Navigate to Radio Station menu ZBE.Right(1)

ZBE.Right(1) ...

# Start radio by selecting a station ZBE.Press(1)

# Start incoming call

MobilePhone.DialNumber("0179xxxxxxxx") #Test: incoming call signaled

if (MMI.getText() != "Incoming Call"): Log.error("Incoming Call not signaled") #Test: radio suspended

if (Audio.getChannel() != Channel.Phone): Log.error("Wrong audio channel")

Listing 6.1. Test script.

In order to automatically transform a task sequence into such a test script, the tester's implicit mappings of tasks to test triggers and system states must be made explicit. In this chapter and the following one we describe how this can be accomplished.

The transformation of an abstract task sequence into an executable test script is performed in two steps:

1. Renement: A task sequence is rened by enriching a task with the corresponding test inputs and system outputs. For example, the task Select Radio Station is rened by adding the IDrive controller inputs that are required to select a station. The renement of task sequences is described in the remainder of this chapter.

2. Test Script Generation: The transformation of a rened task sequence into the test script language. For example, by transforming a task se- quence into a Python script. The transformation of abstract test cases into executable test scripts is the topic of the next chapter.

6.2 Related Work

The concept of mapping abstract models to more concrete ones is a common concept to bridge dierent abstraction levels. There are several publications that use this concept to rene generated test cases.

The work presented by Pfaller et al. is based on the formalization of user requirements in the form of services [PFH+_{06]. They model the renement of}

services into underlying functional models on lower abstraction levels. Based on these relations, they reduce the underlying functional models into the parts that are touched by the service. From these reduced functional models they generate service specic test cases. Their approach is not appropriate for in- stantiating test cases that are generated from task models because our goal is

to generate test cases that cover multiple services. Therefore task sequences cover large parts of the system. Thus reducing the underlying functional mod- els still results in large models with a potentially large number of test cases. Fröhlich and Link map use cases to statecharts using preconditions and post- conditions [FL00]. Based on this relations test cases for each use case are semi- automatically generated from the statecharts using articial intelligence (AI) planning. Their approach is similar to our approach of state-machine-based renement that we introduce in the second part of this chapter. However, task models express dierent task modes resulting from task executions, for exam- ple, when a task is suspended by another task. Such temporal dependencies are not dened for use cases. In order to test these dependencies our approach is to include them during test case renement.