Example for Adaptability Computation

In this section, we demonstrate an exemplary computation of the proposed adaptability metrics. We use two synthetic process models, P M 1 and P M 2, which are implemented in BPMN and depicted in Fig. 6.2.

Figure 6.2.: Example for Adaptability Computation

Both process models have a similar graph structure, but use different elements of the vocabulary of BPMN, which vary in their adaptability score.

Language elements used

The first process model, P M 1 relies on language elements that can hardly be adapted. In particular, these are ComplexGateways, ManualTasks, and a SignalEndEvent. As indicated in its name, the ComplexGateway is the most advanced gateway construct in BPMN. It is used to model complex synchronization behavior that may involve conditional activation based on a number of incoming branches and conditional activation of outgoing branches. Due to its complex configuration, it cannot generally be adapted by another language element of BPMN. The same applies to ManualTasks, which serve as unique placeholders for actions performed without automatic support. For the final SignalEndEvent, semantically equivalent alternatives that represent ordered termination and produce a notification do exist. For instance, a MessageEndEvent could be used to implement similar behavior. In contrast, P M 2 uses elements that can be considered as more adaptable. These are ParallelGateways, a SendTask, a ServiceTask, and a MessageEndEvent. For the ParallelGateway, straight-forward alternatives, such as an InclusiveGateway that is configured to activate all subsequent control-flow branches, do exist. Message-related elements in BPMN are duplicated in the form of message-related tasks and message-related events and, as a result, several alternatives are available for the SendTask and the MessageEndEvent. Also for a ServiceTask, alternatives that do trigger the execution of an application or service, such as a ScriptTask, do exist.

Metric

values The adaptability metrics should consider P M 2 as more adaptable than P M 1. This is clearly the case, which can be seen in the metric values listed in Table 6.125. The table lists the values for the binary adaptability metric with a threshold set at 60% and the weighted metric. In both cases, the metric values for P M 1 are much lower than for P M 2, with 0.17 compared to 0.50 for the binary metric and

0.21 compared to 0.55 for the weighted metric. This demonstrates that both metrics measure what they are intended to.

Table 6.1.: Adaptability Metric Values for the Example Process Model AMbin. AMweighted

P M 1 0.17 0.21

P M 2 0.50 0.55

6.2. Theoretical Validation

In this section, we validate the binary and the weighted adaptability metrics theoretically. The starting is the clarification of the measurement-theoretic properties of the metrics in Sect. 6.2.1. Thereafter, we discuss construct validity in Sect. 6.2.2.

6.2.1. Measurement-Theoretic Validation

Similar to the portability metrics presented in Chap. 4, we classify the metrics defined here as complexity metrics. Since the metrics are computed by the arithmetic mean, we deal with normalized complexity metrics, or densities of complexity, with the same restrictions and properties as before. Complexity metrics should fulfill the properties of non-negativity, null value, symmetry, monotonicity, and additivity [69].

Non-negativity: Complexity metrics should yield non-negative values. Since the adaptability score is defined as the cardinality of a set, it is always non-negative. Moreover, adaptability degrees are mapped to the interval of [0, 1]. The arithmetic mean of a set of values in this interval, i.e., an adaptability metric as defined in Def. 34, is always non-negative.

Null value: The complexity of an empty system should be zero. In our case, an empty system corresponds to a process model p with no elements, i.e.,

p = ∅. For this case, AM (p) is defined as zero in Def. 34.

Symmetry: The labeling for representing the relationships among system el- ements should not affect the metric value. Here, the labeling refers to the ordering of elements in the process graph. The reordering of elements in the process graph, without altering control-flow semantics, is possible for a variety of elements, for instance in the case of parallelism. In the notation of [69], symmetry means that two process models p =< E, R > and p0 =< E, R0 > with identical elements E and control-flow semantics

but different element ordering R and R0 should have the same metric value. We compute adaptability on a per-element basis through the adaptability

degree. This degree is fixed independently of an elements position in the process graph, so symmetry holds: AM (p) = AM ({e1, . . . , en}) = AM (p0)

Monotonicity and Additivity: When two unrelated parts of a system are taken together, the resulting metric value should at least be equal to the sum of the combined parts. This means that complexity metrics should be additive and monotonous. Additivity holds for adaptability degrees, but not for the metrics, due to normalization. We apply the arithmetic mean to achieve a normalization with respect to the size of a process model and adding up the mean values of two process fragments is not meaningful. Hence, additivity in the sense of [69] does not hold for reasons of normalization. Nevertheless, adaptability metrics are still monotonous. For instance, let

p1 = {e1, . . . , ei} and p2 = {ej, . . . , en} be two unrelated parts of process

model p:

AM (p) = AM (p1∪ p2) = AM ({e1, . . . , en}) = (6.6)

AD(e1, ) + . . . + AD(ei) + AD(ej) + . . . + AD(en)

N_|p1_∪p2_|

≥ min(AM (p1), AM (p2))

The metric value of two process fragments taken together cannot be lower than the metric value of the less adaptable of the two fragments and therefore the metrics are monotonous. This is the same result as for the portability metrics, discussed in Sect. 4.3.2.

To summarize the above discussion, adaptability metrics are normalized complex- ity metrics, or densities of complexity, which fulfill the properties of non-negativity, null value, symmetry, and monotonicity.

6.2.2. Evaluation of Construct Validity

The attribute, [70] to be measured in the current chapter is adaptability and the measurement instrument remains the metrics suite, prope.

Purpose of the metrics: The purpose is the facilitation of private self-assessment and improvement, and the information of developers and system adminis- trators about the adaptability characteristics of a process model. When having to port a process model, the metrics can help to make the decision whether it is worth to invest in its adaptation.

Scope of the metrics: The scope of the metrics is a single project of one work- group. The metrics are applicable during and after development.

Measured attribute: The metrics try to measure the adaptability of a process model, the ease with which the elements of a given process model can be modified at design-time without changing the execution semantics of the model.

Natural scale of the attribute: The scale of the attribute is intrinsic to the attribute itself and independent of the way in which we try to quantify it. At the current time, we cannot tell what the scale of adaptability is, but it can reasonably be assumed that process models differ in their adaptability in a way that allows us to construct a natural ordering. Hence, the attribute of adaptability, at the very least, has an ordinal scale.

Natural variability of the attribute: We do not know in which ranges adapt- ability typically varies. Being a technical attribute inherent to a process model, we know that it is not subject to variation common for human attributes, such as developer productivity depending on the time of the day. Definition of the metrics and the instrument: All the metrics, the functions

that assign values to the attribute, are formally defined in Sect. 6.1.1. Adaptability scores of elements are fixed values and obtained through counting. Adaptability degrees and metrics are computed based on this. The measurement instrument is a plugin for our metrics suite in which we implemented the metrics computation. The structure of this plugin is explained as part of the practical evaluation in Sect. 6.3.

Natural scale of the metrics: The metrics are defined in Def. 34 on an interval scale of [0, 1].

Natural variability of the measurement instrument: This question refers to the measurement error of the instrument, in our case a static analyzer. Since we compute the metrics through static analysis, there is no variability of the instrument for subsequent analyses of the same process model. There is no way in which we can guarantee the absence of errors in our software. For instance, programming errors that impact the metric values the tool produces might exist. We try to limit the amount of errors by open sourcing the tool and making all code available to public scrutiny, by providing an excessive set of unit tests for the tool itself, and by applying techniques of continuous integration and inspection during development. Another source of measurement error are the adaptability scores encoded in the tool. These are based on human judgment and, naturally, we can err in our understanding of the semantics of BPMN elements. We tried to minimize these errors through in-group peer review and by making all scores publicly available as part of the tool implementation. Moreover, it is possible that we did not consider all potential alternatives that exist for a particular language element when judging its score. If this is the case, the element is considered as less adaptable by our tool than it is in reality. As a consequence, the metric values computed by our measurement instrument can be treated as a lower bound of the adaptability of a process model. Relationship of the attribute to metrics values: As discussed in Sect. 6.1, the

adaptability of an element is related to the number of alternatives available to that element. The more alternatives exist for a given language element, the

more adaptable this element can be considered. Based on this assumption, our metrics are directly [171] related to process elements. Given elements with a higher degree of adaptability are introduced into the process model, this will be detected by our measurement instrument and the resulting metric value will increase accordingly.

Side-effects of using the metrics: Similar to the portability metrics, we auto- mated the metrics computation fully. Hence, there is no room for human bias in the measurement instrument.

6.3. Practical Evaluation

As before, the practical evaluation of the adaptability metrics demonstrates the feasibility of their computation and exemplifies their interpretation.

In the next section, we first describe the design and instrumentation of the evaluation. This is followed in Sect. 6.3.2 by an outline of the functioning of their prototypic implementation in a plugin to our metrics suite. Finally, Sect. 6.3.3 describes the results of the evaluation.

6.3.1. Design and Instrumentation

The goal of this evaluation is to analyze real-world process models for the purpose of assessing the metrics and selecting the most appropriate ones.

Goal

statement To meet this

goal, we evaluate software that is finished and available, i.e., we perform an off-line experiment. Using the evaluation, we can validate several quality factors for the different metrics, addressed by the following hypotheses:

1. The weighted adaptability computation improves discriminative power in comparison to a binary computation: The ability to distinguish between different pieces of software, the discriminative power, is an important quality factor of a software metric. We expect that the weighted computation, due to more fine-grained scoring, outperforms the binary computation with respect to this quality factor.

2. The metrics can be used for comparing process models of different size: Another important quality factor of a software metric is the ability to allow for a comparison of programs of very different size. This is often problematic, especially with complexity metrics [184].

3. The metric values are resilient to minor changes in the underlying data: Mi- nor modifications to process code should not result in significant changes to the metric value, since this would imply that the mechanism of computation is unstable.

Objects

of study A practical evaluation needs to be based on real-world process models and, therefore, on a particular process language. In this evaluation, we use process models implemented in BPMN [26]. Therefore, we need concrete data in the

form of BPMN process models. As before, we gathered this data from numerous open source projects with the help of the Open Hub Open Source network. To obtain our primary data set, we queried the network in May 2014 for files that:

1. have the file extension bpmn, bpmn2, or xml,

2. contain the keyword definitions, the top-level element of a BPMN-compliant file, and

3. contain the BPMN 2.0 namespace.

We downloaded and analyzed the resulting files using the plugin for our metrics suite described in the following section. We tried to parse every file downloaded from Open Hub and if we could find a valid definitions element and at least one process element beneath it, we computed the adaptability metrics for it. This resulted in almost three thousand BPMN process models. We re-performed the query in October 2014, to obtain a second version of the same data set at a later point in time. This is necessary for evaluating hypothesis 3.