Implications on Application Portability

As can be seen from the previous sections, there is a high variance among the degree of standard conformance that engines provide. This means that the

standardization target of BPEL and BPMN has not been reached. Research question 1.1 In both cases, research question 1.1 can be answered negatively: Current standards are not enough for enabling the portability of service-oriented and process-aware software among engines. For a conclusive answer, it might be worthwhile to evaluate the standard conformance provided by engines for other standards, such as XPDL, as well. Nevertheless, BPEL and BPMN are among the most widely used process standards today. Hence, a lack of conformance of the implementations of these standards is a significant insight. Furthermore, there is no indication that the situation is different in the case of other standards.

Even if standards fail the standardization target and engines only implement a subset of the standard, the porting of applications might still be feasible. This is the case if engines implement the same subset of the standard. In this case, unsupported features cannot practically be used in any application. Features that are not used do also not present an obstacle for porting15. However, as discussed in Sect. 2.3.3, if each engine implements a varying subset of the standard, a loss of portability will be the likely result.

Guide- lines for portable applica- tions

To examine the size of the overlap in standard conformance for the tested engines, Fig. 3.5 displays different subsets of the BPEL conformance test suite tabled by the amount of engines that support a subset. Fig. 3.6 displays the same data for the BPMN conformance test suite. For the case of BPEL engines,

Figure 3.6.: Number of Engines By Percentage of Passed Tests for BPMN only 21 of the 131 tests (16%) are passed by all eight engines under test. For BPMN engines, this number is somewhat higher with 30 of 70 tests (43%). It should be noted that these numbers are not suitable for a quality comparison of the BPEL and BPMN standards, or their engines. The number of BPMN engines we benchmark and the amount of tests of the BPMN test suite is much smaller than for BPEL. For this reason, it is not surprising that we find a higher overlap in features for BPMN engines. Increasing the number of tests or tested engines will likely result in a decrease of commonly supported features for BPMN engines, nearing the values found for BPEL engines.

Only the elements in the subsets that are supported by all engines are actually portable, as intended by the two standards. As discussed in Sect. 3.3.1 and

15_{It is important to keep in mind that this is a practical assumption. It does not apply to} many theoretic approaches, common in academia, that map higher-level specifications and models to standard-conformant process models. For instance, it is common to map process choreographies to sets of interacting processes, implemented in one of the above standards. Examples for such model-driven approaches are [130, 200, 201]. These approaches rely on the portability guarantees made by standards and make use of language features that are not supported on every engine. As the results presented here indicate, this leads to practical issues and any of the above approaches is limited to the engine with which its authors tested it. This restriction has been investigated in [76].

Sect. 3.3.3, these elements are very basic. In both cases, strictly sequential execution of basic tasks can be considered as portable. For BPEL, conditional branching and basic structured looping are portable as well. This is not the case for BPMN. There, pseudo-parallelism can be considered as portable, which does not apply to BPEL. The usage of any other advanced features, such as message correlation or compensation handling will be much less portable. When looking at real-world process models, studies have found that the most frequent language elements used in the models are also basic ones [202]. In the case of BPMN, nearly all of the eight most frequent elements are also supported by all of the engines under test. The implication of these results for portability can be expressed as follows: To build truly portable applications, it is necessary to use only basic language features. In particular, it is necessary to restrict control-flow to sequential execution. When considering realistic application scenarios, in the form of workflow control-flow patterns in Sect. 3.3.2, it becomes evident that only a moderate degree of standard conformance is needed to achieve widespread pattern support. However, one of the crucial features required is support for concurrent activity execution. Only half of the BPEL engines and none of the BPMN engines provides this feature in a standard-conformant fashion. Hence, another implication from the benchmark results is that realistic application scenarios cannot necessarily be executed by a majority of the tested process engines.

Research question 1.2 The common lack of standard conformance in engines can be summarized to frame an answer to research question 1.2: There do exist portability issues in all but the most basic language elements, regardless of the language standard. To built portable applications, it is necessary to limit process models to the usage of a moderate language vocabulary. Contemporary process engines are unable to execute many of the features needed in realistic application scenarios in a portable fashion.

In summary, this section and the answers to research questions 1.1 and 1.2 conclude research question 1. The current state of portability of service-oriented and process-aware software is dire, since the degree of standard conformance in process engines and the amount of commonly implemented features is small. It is likely that many applications developed in practice cannot easily be ported among engines and their users are locked into the systems of an engine vendor. For this reason, the improvement of the portability of service-oriented and process-aware software is a relevant research goal. This is the motivation for the following part of this thesis, that presents a software metrics framework for the measurement and improvement of portability.

Measurement Framework for

Portability

Parts of this chapter have been taken from [77, 170]. As the evidence presented in Chap. 3 demonstrates, the standard conformance of process engines is limited. Consequently, standards-based portability of software artifacts is not guaranteed and the actual portability inherent to a piece of software is unclear. This leads to the topic of the current and the following chapters: Is it feasible to assess the portability of these software artifacts and how can such an assessment be achieved?

We evaluate the feasibility of portability assessment through the construction and validation of a measurement framework. The starting point of this framework in the current chapter is the core characteristic itself, portability. The aim of the chapter is to answer research question 2.1: What are suitable metrics for measuring portability? Together with the following three chapters, this chapter demonstrates the feasibility of measuring portability, answers research question 2, and forms one of the two main contributions of this thesis.

Structure of the chapter The chapter is organized as follows. First, we present our methodology for

measuring portability in Sect. 4.1. Thereafter, we provide formal definitions of portability metrics in Sect. 4.2. Next, the metrics are validated theoretically in Sect. 4.3. Moreover, Sect. 4.4 evaluates the metrics with respect to their practical applicability. Finally, we summarize our findings in Sect. 4.5.

4.1. On the Measurement of Portability

In this section, we first discuss crucial considerations for measuring software portability, followed by the definition of a degree of severity with respect to portability. We outline design decisions made in this definition and present a running example used throughout the chapter.

As defined in Sect. 2.3.2, portability is the “degree of effectiveness and efficiency with which a software product can be transferred from one software environment to another ”. This broad definition includes all subcharacteristics of portability as defined by the SQuaRE model. In this chapter, we focus on the feasibility of the porting, regardless of adaptation or replacement. Hence, we reduce the scope of the quality characteristic to direct portability:

Definition 17: Direct Portability

Direct portability is the degree of effectiveness and efficiency with which a software system can be transferred from one software environment to another, without the need for adaptation or replacement.

The goal of assessing direct portability is to determine how easy it is to take an existing piece of software to another execution environment with the purpose of executing it there. The viewpoint taken is that of a developer or operator, who has to perform the porting and possibly modify source code and configuration settings. Portability is perceived as an inherent property of an application that can be assessed independently of a concrete target environment.

Even though portability is a central part of most software quality models, e.g. [53–59], it is hard to quantify with justifiable effort [203]. In general, it is measured by contrasting the effort required for porting a piece of software to the effort of rewriting it from scratch. Assessing this effort empirically is difficult. To enable an automatable and objective assessment, as indicated in [54, 55], direct portability is computed based on lines of code. The number of portable lines of code is compared to the size of the overall code base. Although this principle is quite simple, it is nontrivial to distinguish automatically between portable and nonportable lines of code. Moreover, such a computation is coarse and its meaningfulness is limited. Any two lines of code have the same weighting, although their criticality might be vastly different.

Our goal is to provide a more accurate measurement and assessment of portabil- ity by taking into account domain knowledge of service-oriented and process-aware software. We map the basic mechanism for computing application portability to service-oriented and process-aware software and extend it by a definition of further metrics that consider the typical characteristics of this kind of software. Additionally, we enrich the computation with empirical data on language support in engines, obtained through the engine benchmarks described in Chap. 3. As mentioned before, we view portability as an inherent property of an application that can be computed, although a target engine on which the application should be executed is not yet known.

Porta- bility metrics based on standard confor- mance

As argued in Sect. 2.3.3, software portability is strongly tailored to the stan- dard conformance of execution environments. Only program elements that are supported by a majority, if not all, of the environments can be considered to be portable. As a consequence, the measurement of the portability of service- oriented and process-aware software should take process engines into account. If all engines support all of the specified language elements in the same manner with respect to semantics, then any compilable program will be portable to any engine. Thus, there are no portability issues. As demonstrated through the benchmark in Chap. 3, this is not the case. Each process engine, regardless of the language, supports a specific subset of the language elements. On the one hand, there is a basic subset of the total language that is supported by every engine. On the other hand, several language elements are limited to a subset of engines, causing portability issues. The more engines support a language element, the more portable it can be considered. The fewer engines support a language element, the more severe it is with respect to portability. This severity should be taken into account when computing portability metrics.