Research Methodology and Structure

The research methodology used to answer the research questions and the structure of the thesis is depicted in Fig. 1.1. The center of the figure shows the methodology

Figure 1.1.: Map of the Thesis

applied and the different steps of the work. Furthermore, it describes how the different steps of the methodology relate to the research questions posed in the previous section, i. e., it explains through which activities the questions are answered. Finally, the figure also shows the relation of the steps in the methodology to chapters of this thesis and explains where the answers to the different research questions can be found. In the following, we explain each of the different methodological steps and their relation to the research questions and thesis structure.

To enhance the readability of the thesis, we streamlined the structure of the different chapters, in particular of Part II. There, all chapters follow a certain template when it comes to metrics definition and validation. The application of the same template for every chapter is intended to facilitate reading comprehen- sion. Moreover, we added margin notes to this section and the remaining parts of the work to guide the reader throughout the document.

System character- istics

We begin by defining the characteristics of the systems that are the objects of study and lay the methodological foundations. The central characteristics

of service-oriented and process-aware software that are relevant to the topic of this work are identified through a discussion of the literature. This includes the foundational theory and technological background of, both, process-aware and service-oriented systems, as well as a closer description of their synthesis. We explain prevalent architectural styles of service-oriented systems in Sect. 2.1, i. e., Web Services [15], REST [16], and microservices [17]. The characteristics of process-aware information systems are described in Sect. 2.2, along with a sum- mary of the languages used in the practical part of this work, i. e., BPEL 2.0 [27] and BPMN 2.0 [26]. Sect. 2.2 also explains how process-awareness and service- orientation can be combined for the task of service composition. This forms the content of the first part of Chap. 2. Thereafter, Sect. 2.3 provides an overview over several software quality models, which is the basis for the selection of the ISO/IEC SQuaRE model [53]. We take a closer look at the relation between portability and standard conformance, and motivate the necessity of evaluating it. Furthermore, we describe the advantages and potentials of software quality measurement, i. e., the reason why we design a measurement framework for porta- bility. Finally, we discuss the theoretical foundations for defining, validating, and evaluating software metrics in Sect. 2.4 and outline the architecture of our prototypic implementation of a software metrics suite. In summary, Chap. 2 lays the theory for answering research question 1 and its subquestions and presents the approach for answering research question 2 and related subquestions.

Portabil- ity issue identifica- tion Research questions 1, 1.1, and 1.2, regarding the current state of application portability and the sufficiency of standards, are conclusively addressed in Chap. 3. This chapter demonstrates that the portability of process-aware and service- oriented software is a real practical problem and provides evidence for it. We begin by listing requirements for a standard conformance benchmark in Sect. 3.1. Following this, we describe the design of the benchmark, including the systems under test, the benchmarking tool and the benchmarking test suites in Sect. 3.2. Through the benchmark, a variety of problems can be identified and an overview of the current state of the art can be given, thereby advancing the state of knowledge. These results are discussed in Sect. 3.3.

Metrics deriva- tion and evalua- tion Each of the chapters of Part II, Chap. 4 to 7, focuses on a particular part of the measurement framework, as prescribed by the software quality model defined in the ISO/IEC SQuaRE methodology [53], and derives, defines, validates, and evaluates a set of metrics. Each of the chapters uses a similar structure for defining and validating the metrics, following common metrics derivation processes, like proposed in, e. g., [67, 68]:

1. Define the goal of the measurement, by specifying its purpose, the objects of study, the quality focus and context, and the viewpoint taken.

2. Specify the questions to be answered by the metrics and experimental hypotheses.

3. Define and validate the metrics.

In each of these chapters, we first state and motivate the measurement goal as defined above and the question the metrics should answer. Thereafter, we

derive and formally define a number of metrics based on the existing literature and the problem domain and state their type and scale. This is followed by their validation according to the validation approach described in Sect. 2.4. In particular, this includes their theoretical validation, formally and informally, with respect to measurement theory [69] and construct validity [70]. Thereafter, we perform a practical evaluation, as described in Sect. 2.4.3. Alongside the practical evaluation, we also state and evaluate a number of hypotheses. Chap. 4 focuses on portability, Chap. 5 on installability, and Chap. 6 on adaptability. For replaceability, dealt with in Chap. 7, we deviate slightly from this methodology. The reason for this is that a plethora of metrics for measuring replaceability is already available and has been validated theoretically and practically. To avoid the duplication of this work, we first perform a review of and a selection from the body of existing metrics. Thereafter, we propose an extension to their computation tailored to the focus of this thesis, and evaluate this extension. Discus-

sion of limita- tions

The thesis is concluded by a discussion of related work in Chap. 8 and a summary of contributions in Chap. 9. The final chapter also points out limitations of our approach and open problems that offer areas of future work.