Subsumption Relations

In software testing, coverage criterion X subsumes coverage criterion Y if every test suite satisfying X also satisfies Y [111]. Similarly, coverage criterion X for execution traces subsumes coverage criterion Y if every set of execution traces satisfying X also satisfies Y . Figure 6.4 shows the subsumption relation between our proposed coverage criteria.

The Intra-LC criterion is the weakest criterion among the others. This is because the coverage scope of this criterion is the entity of an individual abstract service using one execution plan. Therefore, the Intra-LC criterion does not subsume any other criterion.

A set of execution traces satisfying the Inter-LC criterion covers all entities of an individual abstract service when using all execution plans. Thus, the entity required by Intra-LC criterion is part of the super set required by the Inter-LC criterion. Therefore, the Inter-LC criterion subsumes the Intra-LC criterion. Moreover, since the Inter-LC criterion focuses on one individual abstract service and not the whole SC, the Inter-LC criterion does not subsume

Figure 6.4: Subsumption Relation between the Coverage Criteria

any of the global criteria.

The Intra-GC criterion is weaker than its peer the Inter-GC criterion. The scope of coverage required by the Intra-GC criterion is only one execution plan. Whereas, the Inter-GC criterion goes beyond one execution plan and scopes out for the whole set of execution plans. A set of execution traces satisfying the Inter-GC criterion covers all entities of a service composition when using all execution plans. Therefore, the Inter-GG criterion subsumes the Intra-GC criterion. However, the converse does not hold. The Intra-GC criterion does not subsume the Inter-GC criterion.

Since the global criteria consider the whole service composition, both the Intra-GC criterion and the Inter-GC criterion subsume the Intra-LC criterion. Moreover, a set of execution traces satisfying the Inter-GC criterion covers all entities of a particular abstract service, thereby subsuming the Inter-LC criterion for all abstract services of the service composition. However, since the Intra-GC criterion focuses only on a single execution plan, it neither subsumes the Inter-LC criterion, nor subsumes the Inter-GC criterion.

6.2

Summary

This chapter presented our coverage criteria for self-adaptive service composi- tions. The proposed criteria exploit execution plans of a service composition and thus include criteria for one execution plan (i.e., intra-plan) and criteria for all execution plans (i.e., inter-plan). The proposed criteria scopes out for one abstract service and for the whole service composition. Additionally, we presented algorithms for measuring percentage of achieved coverage according to each of the criteria. Finally, we discussed the subsumption relation between the criteria.

Online Test Case Selection and

Prioritization

This chapter presents our approach for online test case selection and prior- itization techniques. The intention is to select and prioritize test cases for re-execution in cases where coverage of a service composition is insufficient.

Our test case prioritization goal is to achieve coverage rate of the coverage criteria presented in Chapter 6 faster by combining runtime monitoring and online testing. In order to consider potential runtime monitoring coverage, our approach exploits information about the time it takes to execute test cases and the usage model of service composition.

7.1

Online Test Case Selection

In cases where the achieved coverage of the selected coverage criteria is insuf- ficient, test cases are selected from a repository of test cases (see Chapter 4), and are prioritized for re-execution.

Test case prioritization is concerned with finding an order for the execution of test cases to achieve a certain goal. According to Rothermel et al. [95], there are many possible goals of prioritization including fast achievement of coverage which we target in our approach, Rothermel et al. state:

“Testers may wish to increase the coverage of coverable code in the system under test at a faster rate, thus allowing a code coverage criterion to be met

earlier in the test process”. 85

Following this goal, prioritization techniques aim to schedule the test cases of a test suite such that code coverage is achieved at the fastest possible rate to reach 100% coverage soonest [65]. Alternatively, prioritizations aim to schedule the test cases of a test suite in a way to ensure the maximum possible coverage is achieved by a pre-defined cut-off point.

An increased rate of coverage provides earlier feedback and evidence about the quality of the system or the service composition (i.e., whether quality goals have been met). An increased rate of coverage is typically correlated with an increased rate of fault detection, which allows triggering adaptation of the service composition earlier.

There are several criteria for measuring the coverage rate of a prioritized test suite, the metrics used depend on the coverage criterion selected. For measur- ing the coverage rate of a prioritized test suite, we adapt the metrics defined by Li et al. [65] for block, decision, and statement coverage. For instance, consid- ering statement coverage, Li et al. used APSC (Average Percentage Statement Coverage). APSC measures the rate at which a prioritized test suite covers the statements. Li et al. define APSC as the weighted average of the percentage of statement coverage over the life of the suite, as follows. Let a test suite T

containing n test cases that covers a set S of m statements. Let T Si be the

first test case in the order T0 of T that covers statement i. The APSC for order

T0 is given by the equation:

AP SC = 1 − T S1+ T S2+ ... + T Sn

nm +

1

2n,

where APSC values range from 0 to 100 and higher values imply faster (better) coverage rates.