Coverage-based Selection

Coverage-based test selection criteria specify conditions that must be satised by a set of task sequences. They represent certain aspects of a system that the engineer wants to test. A coverage-based test selection criterion derives a set of properties from a given task model. These properties are trap properties that are expressed as LTL formulas. In the following we introduce several coverage criteria for task models.

Task Coverage Criterion. This test selection criterion ensures that each task is performed at least once. This is achieved by generating a set of task sequences from a given TTask model, where each task is performed at least once in one of the task sequences. Although, this criterion seems fairly trivial, it covers feature interactions. A task sequence consists of a set of task model states, and each task model state comprises the current modes of all tasks. Hence, when an active task disables another task the corresponding mode in the task model state is disabled. This information can be used to dene test oracles that test if the corresponding task is really disabled in the SUT. As a consequence, the task coverage criterion covers disabling dependencies as well. However, the task coverage criterion covers only one of the possible disabling scenarios.

For a given task model M = (T, S0, ∆, Φ)the task coverage criterion creates

for each task ti ∈ T an LTL formula that selects a task sequence TSi =

hS0, ..., Sni, where Si= hm0, ..., mniin which ti is performed:

¬♦((ti= active) ∧ ♦(ti= disabled))

Then SPIN is invoked for each of these generated LTL formulas in order to generate a counter-example.

Interruption Coverage Criterion. The interruption coverage criterion ex- plicitly covers all feature interaction scenarios in which one task suspends

5.4 Task-based Test Selection Criteria 67

Enter Destination Enter Country

enable

Check Control Message

Confirm Message Signal Low Tire

Pressure enable supend Enter City enable Enter Street enable Enter Housenumber

Signal Low Oil

Level enable enable Start Guidance

enable

Fig. 5.6. Interruption coverage example.

another active task. A task can be suspended in dierent situations. For ex- ample, the task model in Figure 5.6 describes two tasks, Enter Destination and Check Control Message. Check control messages can interrupt the destination input, which is modeled by the suspend dependency. The interruption can occur after Enter Country, Enter City, Enter Street have been performed. Fur- thermore, Enter Destination is interrupted when the task Signal Low Oil Level or the task Signal Low Tire Pressure is performed. Hence, there are eight dier- ent interruption scenarios that must be selected by the interruption coverage criterion.

First, the interruption coverage criterion generates a set of task triples P that describe a potential interruption scenario:

P = {hti, tj, tpi | ti, tj, tp∈ T }

Such a triple hti, tj, tpi ∈ P describes a potential interruption scenario if ti

and tj are atomic tasks and if there exists a suspend dependency from ti or

one of its ancestors to a task tp which is an ancestor of the task tj:

Φ(ti) = ∅ ∧ Φ(tj) = ∅ ∧ ∃δ ∈ ∆. δ = htx, tp, suspendi

∧ tx∈ closure(ti) ∧ tp∈ closure(tj) (5.2)

In Figure 5.6, example task triples are:

ˆ hSignalLowT ireP ressure, EnterCountry, EnterDestinationi ˆ hSignalLowT ireP ressure, EnterCity, EnterDestinationi ˆ hSignalLowT ireP ressure, EnterStreet, EnterDestinationi ˆ hSignalLowOilLevel, EnterCountry, EnterDestinationi ˆ ...

In the next step, the criterion creates for each task triple hti, tj, tpi ∈ P the

following LTL formula:

¬♦(tj 6= disabled ∧ (tj= active U (tj = disabled ∧ ti6= active

U (ti= active ∧ ti6= disabled

Finally, SPIN is invoked for each LTL formula in order to generate a counter- example. The result is the set of counter-examples that were found by SPIN for the given set of LTL formulas.

The interruption coverage criterion selects for each suspend dependency a corresponding task sequence in a task model. The number of generated task sequences can be restricted by focusing only on feature interactions. This is achieved by restricting the set of task pairs to the ones that involve a feature interaction. Hence, for a given a set of features F = f1, ..., fn, a task pair

hti, tji ∈ P must additionally hold that:

∃fx∈ F. ti∈ fx ∧ tj ∈ f/ x

Enabling Coverage Criterion. The enabling coverage criterion covers all task sequences where a task enables another task. Such an enabling situation is covered by a task sequence in which an enabling task and one of its enabled tasks are performed subsequently. In order to cover all enabling scenarios of the task Enter City in Figure 5.6 we have to create two dierent task sequences:

1. ...→ Enter City → Start Guidance → ... 2. ...→ Enter City → Enter Street → ...

First, the enabling coverage criterion generates a set of atomic task pairs P = {hti, tji | ti, tj∈ T ∧ Φ(ti) = ∅ ∧ Φ(tj) = ∅}

which describe a potential enabling scenario. A task pair hti, tjidescribes a

potential enabling scenario when it holds1_:

∃tp∈ T. tp∈ closure(ti) ∧ tj ∈ enabledBy(tp) (5.4)

In the next step, the criterion creates for each task pair hti, tji ∈ P the

following LTL formula:

¬♦(ti= active ∧ tj = disabled ∧

♦((tj= active ∧ ti= disabled) ∧ ♦(tj = disabled))) (5.5)

In order to select only enabled dependencies between dierent features, the extension of the enable criterion is similar to the interruption coverage crite- rion.

Concurrency Coverage. The concurrency coverage criterion covers all task sequences where two independent tasks are executed in an interleaved fashion. Interleaving of tasks is only possible if they have no temporal dependencies. Thereby, the concurrency coverage criterion covers task sequences where a potential unintentional feature interaction may occur. Figure 5.7 shows the two independent tasks Check Control Message and Move Seat. There are mul- tiple task sequences in which these two tasks are executed concurrently. These tasks are performed concurrently by executing their subtasks in an interleaved fashion. The subtasks of Check Control Message each can be performed directly

5.5 Summary 69

Check Control Message

Confirm Message Signal Low Tire

Pressure enable Signal Low Oil

Level enable enable Move Seat Stop Moving Start Moving enable enable

Fig. 5.7. Concurrency coverage example.

after Start Moving or Stop Moving and vice versa, which results in a total of six dierent task sequences.

First, the concurrency coverage criterion generates a set of atomic task pairs P = {hti, tji | ti, tj∈ T ∧ Φ(ti) = ∅ ∧ Φ(tj) = ∅}

which describe a potential interleaved execution of two independent tasks. The tasks ti and tj are independent if they hold2:

@tp∈ T. tp∈ closure(ti)

∧ tj∈ (suspendedBy(tp) ∪ enabledBy(tp) ∪ disabledBy(tp)) (5.6)

In the next step, the criterion creates for each task pair hti, tji ∈ P the

following LTL formula:

¬♦(t1= active ∧ t2= disabled ∧

♦((t2= active ∧ t1= disabled) ∧ ♦(t2= disabled))) (5.7)

In order to select only independent tasks from dierent features, the extension of the concurrency criterion is similar to the interruption coverage criterion.