Model to Graph Transformation

7.4 Summary . . . 177

7.1

Overview

Previous chapters have presented a pattern-based framework and techniques to sup- port the design of enterprise processes and applications integration solutions based on services. Services are the intermediary architecture elements relating the busi- ness operation to the process-wide application architecture elements. This chapter describes how the proposed framework is evaluated in regard with the thesis ob- jectives and it focus is on the overall framework and its support to architecture modelling activities. Tool support for the framework is also described. The next chapter focuses on the evaluation of the proposed pattern matching and discovery techniques.

7.1.1 Influenced System Quality Characteristics

During the design of service-based systems to integrate enterprise processes and ap- plications there are a number of challenges that influence the quality characteristics

of the final architectural solutions. One part of the evaluation of this work refers to quality characteristics that can benefit from the proposed pattern-based framework. Nowadays enterprise software systems no longer evolve as separate entities but evolve integrated with each other in a complex interrelated system [Land 2003]. Changes are constant and process and application integration systems have to be continually re-designed to meet new requirements. Maintainability is a central char- acteristic in this scenario and it helps the organisations’ capacity to change. For process and application integration systems, the analysis and design stages are pre- dominant in terms of cost and consumed time [Hohpe 2004].

On the other hand, functional characteristics are critical during analysis and de- sign stages. Functional suitability during initial stages of development is a basic and important characteristic in regard with requirements coverage. Also, compliance to process regulations is a core aspect of process integration systems [Daniel 2009], [Kharbili 2008]. Identifying early non-compliance to process regulations can reveal future failures in the implementation of a system [Lu 2008].

Considering the categorisation for quality characteristics in the ISO/IEC 9126 standard and its corresponding update in the ISO/IEC 25000 standard series, the ex- plored quality sub-characteristics in this investigation are derived from functionality and maintainability characteristics. These are functional suitability, functional compli- ance, changeability, analysability, reusability and traceability (the latter is discussed in the context of analysability). AppendixBprovides descriptions of these quality sub- characteristics in relation to requirement analysis and architecture design stages1 _in

the development of process and application integration systems.

7.1.2 Evaluation Strategy

The overall evaluation strategy adopted in this work involves three different ap- proaches,

• Experiments. The core technical contribution of this work are the pattern match- ing and discovery techniques used within the context of the proposed LABAS framework. The framework uses patterns to support the design of services and architectures. In order to evaluate the techniques, an empirical evaluation involving a set of experiments is adopted to explore the effectiveness and effi- ciency of the proposed algorithms for pattern matching and discovery. Chapter

8describes the experimental setup and results.

1_{According to ISO/IEC 25010 std., at the earliest stages of development, only resources and pro-} cesses can be measured. When intermediate products (specifications, source code, etc.) become avail- able, these can be evaluated by the levels of the chosen internal measures. These measures can be used to predict values of the external measures.

7.1. Overview

• Case study & ALMA. An scenario based method is used with two case studies to assess the proposed pattern-based framework with regard to its benefits to maintainability. The ALMA method [Bengtsson 2004] was selected among other scenario based methods such as ATAM [Kazman 2000] because it focuses on maintainability and it can be used at various stages of development. This work is centred in the initial stages of development, when domain models are analysed and services are designed. The case studies and scenarios used with the ALMA method provide rich examples for analysing the sub-quality characteristics of interest in regard with the use of patterns and modelling support in the context of the proposed architectural framework. Details on the ALMA method and its use with the case studies are provided in Section7.22_.

• Interviews. A complementary assessment based on interviews supports the evaluation process horizontally. Based on the interviews, the view of a num- ber of practitioners from industry regarding model-based analysis and design techniques is explored. Chapter 9 describes an overview of the interviewing process and a discussion of the results.

Expected benefits for end users and organisations. Beneficiaries of an organised framework and pattern-based techniques to address process and application integra- tion are those end users whose roles are business analysts and IT architects, possibly broadening the range to other related roles such as enterprise architects and solution architects [Tandon 2007]. Inexperienced end users could benefit from the proposed tools providing a medium to reuse design knowledge in the form of patterns. Pattern matching and discovery could facilitate the identification of similar (and possibly re- dundant) designs. As a consequence of identifying similar designs, experienced end users could consider the use of automated tools to facilitate the benchmarking of their solutions. At an organisational level, benefits of using an organised architec- tural approach (framework) and design reuse support (pattern-based techniques) are expected from reducing the complexity of integration solutions and its associ- ated costs regarding maintainability. Inefficiencies such as unnecessary use of time from domain and process experts due to lack of analysis tools, re-working of service designs due to redundances and misalignment between process and software levels are issues that could be addressed with the help of the proposed approach.

2_{Note that in order to follow the same sequence of how the framework (first) and the techniques} (later) were introduced in previous chapters, the ALMA method and its use to evaluate the LABAS framework are presented first - in this chapter - and the experimental evaluation of the central con- tribution (pattern matching and discovery techniques) is presented later in Chapter8. This ordering represents how this work went from general and contextual aspects to detailed and central aspects of the contribution.

7.1.3 Specific Challenges, Solutions and Evaluation Methods

This section refers to the addressed challenges (problems), proposed solutions and adopted evaluation methods. Table7.1provides a summary that includes references to (sub-)quality characteristics affected by these problems. The targeted quality char- acteristics are those early mentioned in Chapter1 (hypothesis), i.e., maintainability, functional suitability, functional compliance and traceability. Throughout the chap- ter, instead maintainability, derived sub-characteristics are analysed: changeability, analysability and reusability. Traceability is discussed in the context of analysability.

Table 7.1: Summary of problems, proposed solutions and evaluation methods

Challenge QSC (∗) Solution (LABAS) Evaluation method

ALMA Experiments Interviews

P1 Suitability, Framework X - X Analysability Changeability P2 Analysability Changeability Traceability support X - X

P3 Reusability Pattern support X X X

P4 Compliance Pattern support X X X

P5 Reusability Com- pliance

Pattern support - X X

(∗) : Affected quality sub-characteristics.

P1 : Separate models for process, domain and architecture descriptions. P2 : Misalignment between process and architecture levels after changes. P3 : Ineffective or inefficient use of design knowledge.

P4 : Lack of automated support for checking regulatory process compliance. P5 : Undesired time consumption and errors due to manual efforts during

pattern identification.

P1 : Separate models for process, domain and architecture descriptions

• Problem description. According to [Kazman 2000] architectures have to be well documented, it includes static and dynamic views and the use of an agreed-on notation that all stakeholders can understand with a minimum of effort. These requirements are difficult to achieve if the architecture spans several processes across an organisation(s) and involve different applications with heterogenous architectures. This situation can be frequent for process and application inte- gration scenarios [Linthicum 2000], [Johannesson 2001]. A basic requirement in this scenario is to have an integrated view of the business operation (pro- cesses) and its supporting application architecture. Maintaining different non- connected models for all elements involved in an integration problem puts in risk the adequate satisfaction of integration needs and correctness of imple- mentation [Johannesson 2001], thus affecting functional suitability. Also, more

7.1. Overview

time is required to analyse what are the consequences over other elements in different layers, thus influencing maintainability.

• Proposed solution. A modelling framework organised in a layered architecture is presented in Chapter 3. The layered architecture includes three main lay- ers encompassing process and domain models, and application and service architectures.

• Affected (sub)-characteristics. Functional suitability and maintainability (analysability and changeability).

• Evaluation method. The Architecture Level Modifiability Analysis (ALMA) method [Bengtsson 2004] is considered to evaluate changeability and analysability sub-characteristics derived from maintainability. Suitability is demonstrated in a case study. The ALMA method and case study are pre- sented in Section7.2.1.

P2 : Misalignment between process and architecture levels after changes

• Problem description. The problem of misalignment between business and soft- ware levels has been investigated since decades and from different perspectives [Luftman 1999], [Henderson 1993]. From the modelling perspective, a funda- mental aspect is the capacity of interrelating the different models involved in both, business and software levels [Lankhorst 2005]. When changes at any of the levels occur, their impact on other layers has to be assessed. Problems associated to unknown effects can result in high costs for re-design and imple- mentation [Chen 2005].

• Proposed solution. Explicit modelling of dependencies and active change man- agement across layers of LABAS framework (Chapter3).

• Affected (sub)-characteristics. Maintainability (analysability and changeability). • Evaluation method. Architecture Level Modifiability Analysis (ALMA) method

[Bengtsson 2004].

P3 : Ineffective or inefficient use of design knowledge

• Problem description. Each time an integration project takes place, analysis and design activities initiate the development. Understanding the enterprise con- text and the problem domain associated to the integration project is the most complex and time consuming part of the process [Linthicum 2000]. Reusing proven design solutions like patterns has been widely promoted as a medium to reduce costs and development time while benefiting design quality – see e.g., [Gamma 1993], [Buschmann 2007], [Barros 2007], [Zdun 2007b]. An inefficient use of design knowledge in the form of patterns could reduce the potential cost

savings of using proven designs, including savings due to reduced efforts of development and testing [Vokac 2004], [Buschmann 2007]. If the conceptuali- sation and infrastructure to create, manage and reuse patterns does not exist, the previously mentioned benefits are more difficult to achieve. Automated support for architectural design has a number of contributions for at least two decades [Shaw 1996a]. With the emergence of new architectural approaches such as service-based architectures and the increase of abstraction levels mov- ing traditional software integration support (EAI tools) closer to business lev- els (BPM tools) [Hill 2009], [Oracle 2008], automated support becomes more specialised and sophisticated, including identification of process artifacts to facilitate reuse and change management.

• Proposed solution. A framework for enterprise process and application integra- tion that allows management of design knowledge in the form of patterns. Patterns can be at process model, and service and application architecture lev- els. A family of techniques is proposed to match and discover patterns in graph-based representations.

• Affected (sub)-characteristics. Maintainability (reusability).

• Evaluation method. A case-study based approach is adopted. It uses the Ar- chitecture Level Modifiability Analysis (ALMA) method [Bengtsson 2004] with scenarios that demonstrate differences in reusability. Techniques providing au- tomation to match and discover patterns are assessed through an empirical evaluation. Description and results are provided in Section8.

P4 : Lack of automated support for checking regulatory process compliance • Problem description. Business process compliance has become a relevant con-

cern for organisations since legislative and regulatory environments have in- creasingly been introduced [Ghose 2008]. Process and application integration projects cover both process and the underlying software layers. Ensuring com- pliance at business process level is a crucial feature for process-centric inte- gration systems. Compliance has traditionally been achieved through efforts performed by a auditing experts with poor or nonexistent automated assis- tance [Kharbili 2008]. Manual efforts for checking compliance and lack of tech- niques and tools supporting this task may introduce errors and significant time regarding regulatory process compliance. Errors in turn, can introduce weak- nesses that can result in fraud, organisational misconduct and loss of organi- sational reputation [Daniel 2009].

• Proposed solution. As one possible alternative to improve the lack of automated support for checking regulatory process compliance, the family of pattern

7.1. Overview

matching techniques presented in Chapter 5 is used to check if process pat- terns defining regulations over the operation of businesses are completely or partially instantiated, or if they are not presented at all.

• Affected (sub)-characteristics. Functional compliance.

• Evaluation method. Using the case study presented in Section7.2, process com- pliance is discussed in relation to hypothetical regulations formulated as pro- cess patterns. Techniques providing automation to match and discover process patterns are assessed through an empirical evaluation. Description and results are provided in Chapter8.

P5 : Undesired time consumption and errors due to manual efforts during pattern identification

• Problem description. As mentioned previously, increased time consumption and susceptibility to errors due to services design can occur due to manual ef- forts during the analysis of dependencies in elements from different models associated to the integration problem, identification of services and process abstractions (patterns) or when checking compliance to process regulations. Automated support for architectural design has a number of contributions for at least two decades [Shaw 1996a]. With the rising of new architectural ap- proaches such as service-based architectures and the increase of abstraction levels moving traditional software integration support (EAI tools) closer to business levels (BPM tools) [Hill 2009], [Oracle 2008], automated support be- comes more specialised and sophisticated, including identification of process artifacts to facilitate reuse and compliance management.

• Proposed solution. A family of pattern matching and discovery techniques to assist the definition of new services and compliance to process regulations is presented in Chapters5and6.

• Affected (sub)-characteristics. Maintainability (reusability) and functional com- pliance.

• Evaluation method. Techniques providing automation to match and discover patterns are assessed through an empirical evaluation. Description and results are provided in Chapter8. References to the use of these techniques in scenar- ios used with the ALMA are also provided.

7.2

ALMA-based Analysis of Case Studies

The ALMA method is used with two case studies that include a number of differ- ent scenarios that provide rich examples for analysing changeability, analysability, reusability, traceability, functional compliance and suitability with regard to the use of patterns and modelling support in the context of the LABAS framework. The case studies and scenarios capture models, model changes and abstractions in the form of patterns in different layers of the integration problem (process model, service ar- chitecture and application architecture layers). The case studies capture a scenario of integrated financial network services from an application perspective and a scenario of a typical process in the e-commerce domain where customers and businesses in- teract. Note that the cases have limitations with regard to the need of an empirical justification where scenarios are confirmed by a relevant number of analysts or ar- chitects and concrete implementations can be controlled. Beyond these limitations, the aim of considering these scenarios has been to represent common situations of processes and applications integration problems in organisations.

7.2.1 Architecture-level Modifiability Analysis Method

ALMA method is a scenario-based method designed for predicting mainte- nance efforts, assessing risk and comparing different candidate architectures [Bengtsson 2004]. The ALMA method consists of five steps3, described as follows.

• Set goal: setting the analysis goal;

• Architecture description: giving a description of the relevant parts of the software architecture and their configuration;

• Elicit scenarios: finding the set of relevant change scenarios;

• Evaluate scenarios: determining the effects of the set of scenarios; and • Interpret results: drawing conclusions from the analysis results.

Even though the main target of the ALMA method is to assess modifiability of a system, the method is adopted to analyse a closely related quality characteristic: changeability. Most of the aspects covered by the definition of changeability in Sec- tionB.4.1overlap with the definition of modifiability in ALMA, that is defined as the ease with which a software system can be modified to changes in the environment, requirements or functional specifications.

On the other hand, analysability, as a characteristic that benefits the easiness to modify a software system, can also use the change scenarios derived in ALMA. Moreover, these scenarios provide rich examples to assess functional suitability and

3_{Note that while performing the analysis, sequentiality of steps is not strict and it is often necessary} to iterate over various steps.

7.2. ALMA-based Analysis of Case Studies

compliance. The case studies presented in the next sections use the steps defined in the ALMA method to assess the benefits of the proposed framework and techniques to changeability, analysability, suitability and functional compliance.

Assumptions in this work. A number of assumptions are considered when ap- plying ALMA to the case studies in the next sections. These involve the existence of models describing the different layers of the integration problem, that all models can be translated to a single graph-based notation, and the existence of a mecha- nism to access models even though they could be located remotely in a distributed environment.

Note that process models may have been created manually (during traditional documentation activities) or automatically by using tools to mine process logs, e.g., [Aalst 2007], [Bae 2006], [Greco 2005]. The only conditions for these models are the last two listed above. Also, note that the previous assumptions are valid for future case studies involved in further research for assessing the proposed framework and techniques, which can be applied in early stages of analysis and design for service- based process and application integration systems development.

7.2.2 Loan Management (LM) Case

7.2.2.1 Analysis goals

The analysis goal is to get a comparative analysis of the costs of modifying the LM process and its supporting systems. The emphasis is on analysing the relative bene- fits for the proposed framework and a manual-based approach regarding analysabil- ity, changeability and functional compliance support. The analysis is a post-mortem analysis and therefore there is no need to normalise the weights of each change scenario in ALMA [Bengtsson 2004]. The change scenarios involve modifications resulting from the improvement of a bottleneck activity that affected the operation of the business, the incorporation of new process regulations and technological up- dates. Changes resulted in the re-design of the Loan Management process and its underlying software.

7.2.2.2 Processes and software architecture

This case study looks at a scenario of integrated financial network services from an application perspective. The main process involved is a traditional loan manage- ment process. This is a basic banking operation and it represents a good example of a process in the financial domain. The aim in this case is to integrate the process’s supporting software to facilitate the tasks performed by bank agents, specially those

considered to be a bottleneck. Also, internal rules for complying with loan manage- ment regulations would require modifications of some of the activities in the Loan Management process.

Figure7.1shows three business actors and their participation in a simplified loan management process. The process starts when a client requests a loan by email. A bank agent from a call centre calls the client to present him an offer. To provide the offer the agent needs to obtain client data, calculate the amount of loan offer and to call the client to explain the conditions of the loan (top of Figure7.1). In subsequent steps of the process, the agent registers relevant information regarding the client’s acceptance respect to the offer. If the client accepts the offer and the amount of money involved in the loan is sufficient to meet the sales goals of the bank, then a