Practice-driven approach for creating project-specific software development methods

(1)

Practice-driven approach for creating project-speciﬁc

software development methods

Marko Bajec

*

_{, Damjan Vavpoticˇ, Marjan Krisper}

University of Ljubljana, Faculty of Computer and Information Science, Trzaska 25, 1000 Ljubljana, Slovenia

Received 16 December 2005; received in revised form 6 May 2006; accepted 25 May 2006 Available online 3 July 2006

Abstract

Both practitioners and researchers agree that if software development methods were more adjustable to project-specific situations, this would increase their use in practice. Empirical investigations show that otherwise methods exist just on paper while in practice developers avoid them or do not follow them rigorously. In this paper we present an approach that deals with this problem. Process Configuration, as we named the approach, tells how to create a project-specific method from an existing one, taking into account the project circum-stances. Compared to other approaches that deal with the creation of project-specific methods, our approach tends to be more flexible and easier to implement in practice as it introduces few simplifications. The proposed approach is practice-driven, i.e. it has been devel-oped in cooperation with software development companies.

Keywords: Method engineering; Process engineering; Project-speciﬁc method

1. Introduction

It is a common belief among the researchers of the IS community that the use of methods in software develop-ment is beneficial, since it improves the process and its product. Interestingly, investigations in practice show rath-er diffrath-erent picture. It seems that methods are actually underused and what is more their use is not increasing (see e.g. [13,19,20,10,26]). One reason contributing to this rather paradoxical situation isinflexibility, which is a char-acteristic of methods that permits virtually no adjustment to organisation-specific or project-specific circumstances. As stated by Fitzgerald, in software development, methods are not applied rigorously nor uniformly, even when train-ing in their use has been provided. This supports the view that a uniquemethod-in-actionis created for each develop-ment project[10].

In this paper, we discuss how flexibility can be inte-grated into the methods that software development organisations are using in their every-day practice. The approach that we present (Process Configuration Approach or in short PCA) builds on the previous research work known from the literature as Method Engineering. The method engineering deals with the cre-ation of methods that are specifically attuned to the needs of a particular organisation or project. While there were several approaches proposed in the past that sug-gest how to create methods on-the-fly, their use in prac-tice is somewhat low, which seems to be a result of their inherent complexity. In the PCA we tried to avoid the problems that hinder the use of traditional assembly-based method engineering in practice by introducing some limitations. We believe the PCA contributes a very useful architecture that can be used to implement any approach to method engineering that is based on the tai-loring of a single method.

The PCA was developed as a part of the MasterProc

project, aimed at developing a framework and supporting

*

Corresponding author. Tel.: +386 14768 814; fax: +386 14768 705.

E-mail addresses:[email protected](M. Bajec),damjan.vavpotic@

fri.uni-lj.si (D. Vavpoticˇ),[email protected](M. Krisper).

(2)

tools for reengineering software development methods in Slovenian companies.1

The paper is organised as follows. In Section 2 we describe the research approach adopted in our work. Next is the related works section which brieﬂy describes other approaches to method construction for the needs of a par-ticular organisation or project. The core of the paper is in Sections4 and 5. Section4gives a theoretical background of the approach while Section5describes the results of the application of the approach in practice. In Section 6 we compare our approach with other SME approaches and illustrate how the PCA could be used in combination with some of the existing approaches to method engineering. Finally, in Section7, concluding remarks are given.

2. Research approach

2.1. Collaborative practice research

The research approach adopted in the MasterProc pro-ject is founded on theCollaborative practice research(CPR)

[25] that proved to be very useful for projects aimed at understanding, supporting and improving software devel-opment practice. The CPR deﬁnes three goals that are important in software process improvement and suggests activities that help to achieve them. First, to understand the current state of software development (the goal) the interpretation of practice has to be engaged in (the activi-ty). Second, to build new knowledge that can support prac-tice (the goal), normative propositions, such as guidelines, standards, methods, tools, etc. has to be designed (the activity). Finally, to learn what is needed to actually improve practice (the goal), various social and technical interventions have to be employed (the activity). Although the three goals are distinct in nature and can be pursuit in isolation this is not suggested as it would seriously reduce the opportunities to learn about practice.

The CPR takes as the main consideration the difficulty of establishing effective and well functioning relations between research and practice. Ideally, the research process should be tightly connected with practice to get firsthand information. At the same time the research needs to be well structured and managed to produce rigorous and publish-able results. This, of course, is difficult if the project agenda strongly depends on how practice evolves. The CPR recog-nises three basic research approaches that help to take into account the dilemmas how to fulfil the two criteria. These are: Action research, Experiments and Practice studies. Action research is based on strong integration between research and practice. The researcher is involved in

practical situations, which gives him/her optimal access to practice. The most important weakness of this approach is that it makes the research process and findings hard to structure and manage. Better option in this sense are exper-iments, which provide access to practice that are controlled or partially controlled by researchers. In contrast to action research they provide weaker relation to practice. Finally, practice studies present a research approach to study soft-ware development without researchers being actively involved in practice. The practice studies cover different types of study:direct study, e.g. field studies or case studies, andindirect studies, e.g. surveys or interviews.

2.2. Organisation of the MasterProc project

The PCA approach that we describe in this paper has been developed as one of the deliverables of the aforementioned MasterProc project. The aim of the MasterProc was to improve software development practice in Slovenian compa-nies by developing a framework and supporting tools for reengineering software development methods. The project was organised as a collaborative practice research using a combination of diﬀerent research approaches. In this subsec-tion we describe in more detail how the MasterProc was organised. Please note that the development of the PCA was just an activity of the MasterProc project.

In the organisation of the MasterProc project the princi-ples of ageneral learning cyclehave been adopted, i.e. inter-pret current situation, find ways to improve practice, plan and implement improvements, and learn from the actions taken. As described above the CPR supports such learning cycle by the three goals it identifies: to understand the current state of software development, to build new knowledge that can support practice, and finally to plan changes and imple-ment them as necessary. After impleimple-menting the improve-ments, the interpretation of the lessons learned have to take place, hopefully leading into the next learning cycle.

At the time this paper was written, we were still under-going the first learning cycle of the MasterProc project. According to project plan the first learning cycle should be finished till September 2006, while the next is scheduled for the beginning of 2007.2To describe the organisation of the MasterProc project clearly as to present how the devel-opment of the PCA that is described in this paper fits into the whole project we provide detailed descriptions inTable 1. For the timeline of the project seeFig. 1. The activities that were carried out to develop the PCA approach are deliberately described in more detail and are coloured in green. (For interpretation of the references in colour in the text, the reader is referred to the web version of this article.)

1 _{The MasterProc is a research project which is carried out under the}

umbrella of the Centre of excellence. One of the missions of the Centre of excellence is to improve software development methods in Slovenian companies. The project is co-founded by the Slovenian Ministry of Higher Education, Science and Technology, European Commission and the participating Software Companies.

2 _{The timeline and learning cycles of the MasterProc project are}

influenced by the dynamics of the Centre of Excellence. According to the currently known information the Centre of Excellence will finish its first programme period in the end of 2006 and start with the next period in the beginning of 2007.

(3)

Table 1

The research approach

Goal 1: to understand the current state of software development Activities:

for each of the participating companies:

– [A1] Acquire information on existing ways of working (how the company performs its software development projects).

– [A2] Determine the level of ﬂexibility a software development method would require to appropriately support typical projects of the company.

Research approaches:

– interviews and questionnaires. – action research.

Description:to carry out an assessment of the existing state of the art of software development methods in each of the participating software companies several meetings were organised. A combination of interviews and questioners was used to acquire information on how the companies currently develop software. The main objective of the assessment was to determine how socio-technically suitable are the methods for typical projects carried out by each of the software companies and to identify deficiencies that would need improvements. Furthermore, the goal was to identify, for each of the participating companies, how different in terms of the ways of working are the projects it performs. This information indicated what level of flexibility would be required for the company’s method in order to provide appropriate support for its ways of working. The information that we received from the interviews and surveys was complemented by action research. For each of the participating software companies a working team was set up comprising two researchers and two practitioners. The main responsibility of the team was to take part in real projects to get firsthand information on how a particular company is developing software. The practitioners acted as project managers and method engineers, while the researchers were more or less observers.

Goal 2: to build new knowledge that can support practice Activities:

– [A3] For each of the participating companies conduct a research to understand how to resolve identiﬁed deﬁciencies in its ways of working; thus to improve current practice.

– [A4] Conduct a research to understand how to create ﬂexible software development methods.

Research approaches: – literature review.

– experiments and ﬁeld studies.

Description:the objective of the second phase of the MasterProc project was twofold: first, to find out how the existing ways of working in each of the participating companies could be improved, and secondly, to find out a solution for creating flexible methods, i.e. methods that companies would be able to adapt to circumstances of a particular project. The weaknesses of the companies’ software development processes that were identified during the first phase of the project were thoroughly studied. Improvements were proposed in terms of guidelines, methods, tools, etc. and discussed in one-day workshops with each participating company. Some of the improvements were tested through experiments and field studies. For example, in one of the companies it was found out that when iterative development was taken the project managers were unable to manage iterations. To resolve this problem a research was conducted to find out how different approaches, specifically agile methods, suggest managing iterations in iterative software development. The solution that we suggested was first tested in our laboratory and after that in a real project performed by the company. Since the feed back was positive the improvement was scheduled for its implementation.

The second objective of this phase was to make methods of the companies flexible, so that they could be adjusted to project-specific circumstances. This objective was much more difficult to achieve. At first we conducted literature review to study state of the art in the field of method construction and its deployment. The possible approaches to integrate flexibility into the methods were than presented to chief information officers of the software companies. As it was expected, they all disagree with the possibility to integrate flexibility into their methods by using traditional assembly-based approaches to method construction, as they found them to complex and risky. They consented however to capture their methods (before the project all the companies used informal methods) and to develop appropriate tool support that will help them in adjusting these methods to suit particular projects. As a result of this activity the PCA approach was designed together with its tool-support. The PCA was tested in the laboratory environment and also in real settings provided by the participating companies. The purpose of the evaluation was to find out whether the assumptions on which the PCA was based were correct. For the details see Section5.

Goal 3: to learn what is needed to improve practice and to actually improve practice Activities:

– [A5] Conduct a research on improvement strategies.

– [A6] For each of the participating companies identify appropriate strategy to make improvements to their software processes and to introduce PCA approach.

– [A7] Implement improvements.

Research approaches: – literature review. – survey.

Description:once current practice is understood and new knowledge is built to support the practice, appropriate measures need to be taken to actually improve the practice. In the MasterProc project a thorough research was conducted to understand how to measure and specifically how to improve software development practices by considering technical and social characteristics of the ways of working in a particular organisation (see e.g.[30]). One of the results of this research is the set of strategies that helps to implement improvements based on the socio-technical characteristics of the problem domain. For example, in the case described above our goal was to improve the company’s practice in the activity that was responsible for managing iterations. The problem with this activity was not that it would not be adopted by the company’s project managers, but it was technically inappropriate, as it leaded into loosely planed iterations. The strategies for implementing improvements to resolve technical deficiencies are much different than strategies for implementing socially-based improvements.

The implementation of improvements into the participating companies (including the PCA approach) is still under execution. According to our plans it will be concluded in September 2006.

(4)

3. Related works

The research in the software development field has brought about many ideas how to deal with the inflexibility of software development methods. Two of them are a contin-gency factors approachand, specifically,method engineering.

3.1. Contingency factors approach

The contingency factors approach (see e.g.[1,8]) is based on the suggestion that speciﬁc factors of the development context are considered and taken into account when select-ing an appropriate method from the portfolio of methods. The problem with this approach in practice is that an orga-nisation is expected to be familiar with a range of methods from which they choose the most appropriate one based on the contingencies of the current situation. In practice it is at least unusual for an organisation to have a number of methods available and even more unusual for the develop-ers to be familiar with all of them. This problem has been acknowledged among the researchers who suggested a more pragmatic approach [1]. Instead of selecting from the portfolio of methods the method should be wide enough to encompass all the possible situations. This is the idea that we have followed also in our approach.

3.2. Method engineering

Method engineering is another approach to create soft-ware development methods that are speciﬁcally attuned to organisations or projects. In general, the idea lies in the conceptualisation, construction and adaptation of methods and tools for the development of information systems[5]. Method engineering has a long tradition in the systems development research. An excellent review of the past research can be found in the work edited by Brinkkemper et al [4]. For a review of more recent eﬀorts see the work of Ralyte´ et al[27].

Irrespective of its popularity, the method engineering has never been widely acknowledged or practised by soft-ware engineers. In[16], Henderson-Sellers argues that this is because practitioners often (unfairly) view method engi-neering as having a costly overhead and do not take into account the cost and eﬀort in cases when an ‘out-of-the-box method’3is used and found as inappropriate to compa-ny’s business processes. Whether this is true or not, one thing is certain: method engineering lacks examples of practical application in real development practice[12].

A speciﬁc direction in method engineering which shares the same objective as the approach that we advocate in this paper is situational method engineering (SME). In general, SME deals with developing project-speciﬁc methods or

adapting existing ones to speciﬁc project situations (e.g.

[15,24]). The next section describes various approaches to SME.

3.3. Approaches to SME

In the literature, a number of approaches can be found that propose the creation of project-specific methods. One that is probably the most popular is the so-called assembly-based SME, which follows the reuse strategy. In this approach a new method is constructed from the fragments of existing methods. The notion of method fragment was introduced by Harmsen and Brinkkemper[15]who defined it as a reusable part of a method. Fragments can be further categorised into product and process fragments depending on the perspective they cover. Much effort has been put into decomposing existing methods into fragments [4]. Also, different repositories have been proposed for their storage (e.g. [15,27,4]). The method construction using the reuse strategy is, however, far from easy, as the frag-ments have to be first retrieved from the repository, chan-ged if necessary and than assembled together into one consistent and congruent method.

Another approach to SME, known from the literature as theextension-based approach, uses theextension strategy. In this approach, method engineers are provided with extension patterns that help them to identify typical extension situa-tions and provide guidance to perform extensions. In[27], Ralyte´ describes two possible ways to perform extensions: (a) directly through matching extension patterns stored in a library to satisfy the extension requirements, and (b) indi-rectly through first selecting a meta-pattern corresponding to the extension domain and then guiding the extension apply-ing the patterns suggested by the meta-pattern. Karlsson and A˚ gerfalk have, however, criticised this approach for not con-sidering situations that are actually very frequent in practice, i.e. when a method is both extended in some fragments and reduced in others[22]. As a solution they proposed a new method for SME that uses a combination of the cancellation and extension operators. They named itmethod for method configuration(MMC). The MMC differs from the aforemen-tioned approaches also in the fact that it does not deal with modular construction of a method but rather with method tailoring taking a particular method as the starting point. From the literature, it is clear that this approach has been somewhat overlooked by the method engineering research in the past.

Finally, the approach to SME that seems to be a result of the most recent eﬀorts in the method engineering research is theparadigm-based approach [27]a.k.a. evolu-tion-based approach[2]. This approach is founded on the idea that the new method can be obtained either by abstracting from an existing model or by instantiating a metamodel. A new method is then created by ﬁrst con-structing a product model and then process model while for the construction of both product and process model dif-ferent strategies are available.

3 _{‘Out-of-the-box methods’ are presented as ready for immediate use.}

Sometimes they are also called ‘of-the-shelf methods’ or ‘pre-packaged methods’. RUP (Rational Uniﬁed Process) is an example of such a method.

(5)

3.4. How the PCA ﬁts into the existing SME research?

To position the PCA within the existing SME research we will first describe the characteristics that can be used to compare different SME approaches. There are several aspects in which SME approaches differ, but most notably in:

• the point of departure,

• the granularity of method fragments used to create situ-ational methods, and

• the ability to execute.

The point of departuretells what kind of source a SME approach is using to create situational methods. There are at least three points of departure that can be distin-guished: (a) a portfolio of methods, (b) a portfolio of frag-ments from different methods, and (c) a single method. In the first case (a), a portfolio of methods is considered and the method that most matches the project circumstances is selected to be used on the project. An example of the SME approach that uses a portfolio of methods as a point of departure is the Contingency factors approach described in Section3.1. In the second case (b), situational methods are created from the selection of fragments that are stored in some repository. The fragments belong to different methods and may overlap among each other. The SME approaches that fall into this category are difficult as they require fragments from different methods to be assembled together. The assembly-based SME approaches are exam-ples of this category. In the last case (c), the source is rep-resented by a single method (a.k.a. base method). The method is divided into several fragments which are exam-ined according to the project circumstances and extended, omitted or changed if necessary. An example of this catego-ry is the MMC approach.

Another distinguishing characteristic of the SME approaches is the granularity layer at which situational

methods are created. The granularity layer of a particular SME approach tells the precision of the elements the approach works with when it creates new or adapts an existing method. The notion of granularity layer has been introduced by Brinkkemper et al. [6]who defined a classi-fication framework for method fragments. Based on this classification, method fragments can be classified according to three dimensions:perspective,abstractionand granulari-ty. The dimension that seems to be the most important and the main discriminating property of method fragments is the granularity layer at which the fragments reside. At one extreme, the granularity layer could be as high as to cover a complete method, while at its lowest layer it would only cover a small piece of a product that is delivered by a small part of the method or the procedure that tells how to produce this product. The granularity layer has been recently debated to be very important SME characteristic, as it influences the complexity of the method engineering, in particular the method assembly (e.g.[22,31]). If method fragments are presented at high level of abstraction the construction of new methods is easier, as the method engi-neers do not have to worry about the details. Of course, to make this work, the fragments have to satisfy certain crite-ria (for details see [31]). Another issue worth to be men-tioned at this point is that the layer of granularity presents a trade-off between complexity and flexibility. In other words, if the granularity layer of the method building blocks is high, the flexibility in producing new methods is low. In an ideal situation, a SME approach would work with fragments that support the highest possible precision in creating new methods while the consecutive complexity would be resolved by an appropriate tool-support.

The last but not least characteristic that can be used to classify various SME approaches is theirability to execute. We use this characteristic to denote how deeply a particular SME approach goes in describing the concepts, data struc-tures and algorithms to create or tailor a new method. For a SME approach it is of utter importance that it assures the

(6)

created methods are consistent and complete and that they suit best to the project circumstances for which they were created. How the consistency is achieved, how the com-pleteness of the created methods is verified, how the frag-ments are described and retrieved when needed, how the contingency factors are expressed etc. are questions that need answers in order to use the SME approach in real practice. In light of this characteristic, the existing approaches differ a lot. Some of them are described rather superficially while others provide much more details on how the approach actually works.

According to the above characteristics the PCA approach can be classified as follows: it uses a single method as the point of departure (base method), it works on a low layer of granularity (thus providing the highest possible flexibility) and provides sufficient details to execute in real practice. For the construction of situational meth-ods it uses a combination of the meta-modelling and exten-sion/reduction based approaches. As such it shares several commonalities with other approaches to SME, but most notably with MMC. Both PCA and MMC suggest config-uring an existing method rather then assembling fragments from different methods to construct a new one. As we will show in the next section, the PCA introduces some features that simplifies the creation process and might prove useful also in the combination with other approaches (for details see Section6).

4. The process conﬁguration approach

4.1. Overview

The idea that lies behind the PCA is relatively simple and can be explained as follows (see also Fig. 2): for each individual project a specific process configuration (pro-ject-specific method) is created. This is done by selecting components from a method that has been specifically designed for the organisation and thus reflects its actual performance on the projects (base method). The configura-tion is done by processing the rules that tell in what

circum-stances or project situations it is compulsory, advisable or discouraged to use a particular component.

This section is organized as follows: to avoid confusions we will first define the terminology that we use in the paper (Section4.2). Sections4.3 and 4.4will then describe how a base method can be created and represented, and finally, in Section4.5, algorithms for the PCA will be explained.

4.2. Terminology

In our research we try to understand a method also as a social construct. A method is full of philosophy, principles, ideas and points of view of the organisation staﬀ or its users, what explicitly emphasizes its social component. As Cockburn [7]has emphasised, a method is everything we do to achieve a certain result, i.e. a product or services, which are the goal of our work. When talking about sys-tems development this does not merely mean activities that are directly connected to the development (i.e. analysis, planning, etc.) but also patterns of communication, collab-oration and coordination, support procedures, means of communication with the parties involved, rules of decision, etc.

We also distinguish between methods that are informal

and those that areformalised. A method consists on one hand of elements such as procedures, rules, directions, tools, standards, etc. which can be documented either in electronic or classical manuals, and on the other of certain undocumented parts, and above all the knowledge of the organisation members [3]. This is of utter importance for an organiation, because it represents its own values and competitiveness. A method that is used by a particular organisation is typically richer then its’ formalised and doc-umented part. A substantial part of a method is embodied by its users through the knowledge and experience they car-ry. When such knowledge becomes suﬃciently expressible and the tasks in which the knowledge is used suﬃciently repeatable, the formalised part of the method can be enriched with new elements (e.g. models, procedures, guidelines, advices, etc.). Besides the fact that informal

(7)

methods could never be fully formalised, empirical investi-gations show that many software development companies do not have any formalised method at all. Or if they have one, they rarely keep it consistent with their actual perfor-mance on IT projects (e.g. by enriching it with new knowl-edge and experiences). The distinction between informal and formalised methods has been clearly acknowledged also by other researchers. Fitzgerald, for instance, empha-sises that there is often a wide disparity between the official development process and the actual behaviour of develop-ers in practice[9]. This suggests that in each development project a specific method in-action is created which is shaped by the characteristics of the context, the developers and the information system under development as well as the formalised method and the rational and political roles it plays [11]. The significance of the relationship between formalised method and its background understanding has been discussed also in the work of Introna and Whitley

[21]. In their work they actually criticised the idea of method engineering for being based on rather naı¨ve assumption that ‘method is a necessary and suﬃcient requirement for successful information systems develop-ment’. In contrast, they suggest that ‘the successful use of method itself depends on a brother, already present, tacit understanding of the world, an understanding not to be found in any one particular method’.

In the PCA we deal with two kinds of methods, which both are formalised: base method and project-specific method. While a base method is specifically attuned to a particular organisation, a project-specific method is adjust-ed to the neadjust-eds of a particular project. The distinction between a base method and project-specific method is that the former includes all possible elements that might be use-ful in performing organisation-specific projects (i.e. a repository of the organisation-specific process compo-nents), while the later only consists of those elements that are required for a particular project.

4.3. Base method

Base method is probably the most important prerequi-site for the PCA. It is a formal representation of how a par-ticular organisation is performing its projects. The construction of a base method is thus crucial as it presents a foundation for creating project-speciﬁc methods.

The process that supports the creation of a base method is part of the framework for method reengineering. Since its detailed description would be beyond the scope of this paper, we will provide only a brief description of its main activities. More detailed description can be found in[3].

The construction of a base method is a procedure that has to be done for each organisation individually. It starts with the analysis of the existing practice in an organisation and leads into the identiﬁcation of the method components (activities, tasks, products, etc.) that are technically and socially sound and those that are in these respects problem-atical. Possible improvements to the existing practice are

then suggested and discussed with the organisation’s devel-opment team. Once a vision for a new method is developed and accepted, a metamodel for the organisation’s base method is designed.

A metamodel for the organisation’s base method can be developed from scratch, or by using existing metamodels that have been recently constructed to underpin and help formalise methods. Existing metamodels represent a good source for selecting generic concepts for an adequate repre-sentation of the organisation’s base method. When such a metamodel is developed, it is used to help creating the organisation’s base method, i.e. to capture the method fragments. In this process different fragments are captured. Besides fragments that are based on the organisation’s existing practice, previously approved as technically and social appropriate, many new fragments may emerge. These are based on the suggestions for improvements, iden-tified within the analysis of the existing practice. The frag-ments are first classified according to the underlying metamodel and then described using templates. The tem-plates belong to a metamodel and outline how elements of a certain metamodel type should be described.

In the PCA, it is essential that the underlying base method and corresponding rules continuously evolve as a reﬂection of knowledge and experiences acquired through project performance (see Fig. 3). This means that using the PCA new fragments may emerge as a result of situa-tions that are speciﬁc and thus not yet supported by the base method under use. In such cases, additional fragments are captured to describe these new situations, and circum-stances are determined in which these new fragments should be used. In practice, it actually takes some time for a base method to become all-inclusive in terms of pro-viding guidelines for all kinds of situations that may hap-pen in projects that a particular organisation is performing. This phase, in which base method rapidly

(8)

evolves, is called the learning phase. It takes place in the ﬁrst few projects after the PCA is introduced into an orga-nisation. After the learning phase base method becomes more stable and changes on a large scale less frequent.

The idea that it is possible to elicit the ways of working in a particular organisation and to create formalised method (base method) based on the elicited information should not be seen in conflict with the aforementioned find-ings of Introna and Whithly[21]. What they were arguing was that for an effective use in various development situa-tions one single, pre-developed method will probably not be sufficient. The hypothesis on which our approach is based is not against this opinion. We believe however that if for a particular organisation a formalised method is cre-ated directly from its existing practice and knowledge, improved in parts where found inappropriate and than continuously supplemented by new experiences and knowl-edge that developers gain from the method use, such method might prove very useful for the organisation as it can help to perform projects in more consistent and trans-parent manner. Moreover, in this way – by using feed-back from method use to enrich formalised part of the method we take care the new knowledge and experiences from the method use are not lost but shared within the organisa-tion’s development community. An in-depth discussion of the needs, rationale for and philosophy of the method evo-lution that supports our view on the method engineering has been provided by Rossi et al [29]. In their work they suggested how a method rational which they define as a ‘trace of evolutionary method changes and associated method use experiences’ can help to share knowledge of methods between method users and engineers and how

can act as a powerful mechanism to maintain systems development knowledge in an organisation.

4.4. Representation of a base method

For the purpose of the PCA we designed a generic data structure that can be used to capture an arbitrary meta-model. The idea of a generic data structure is to allow method engineers to design metamodels according to their perceptions of how a method should be formally represented.

Fig. 4illustrates the main components of the aforemen-tioned generic data structure, base method and project-spe-ciﬁc method. The classes representing metamodel are: a

metaelement(it can be of two types: content element, such as activity, tool, discipline, role, etc. or process flow ele-ment, such as decision node, join and synchronisation) andmetalink(links between metaelements). By using such a generic data structure, a base method is represented as a structure of instances of the metaelements and metalinks, and a project-specific method as a selection of the elements and links of the base method. According to[28], the Meta-model represents abstraction level 1, i.e. theProcess Meta-level, while both the Base method and Project-specific method represent abstraction level 2, i.e. theProcess model. As mentioned before, a base method encompasses vari-ous situations that may occur when projects are performed. In other words, it comprises a number of elements and their alternatives which describe several possible ways to perform a particular project (In[4], Hares introduces the notion of project paths to describe different ways of per-forming a project). The paths and method structure,

(9)

however, are not fixed in the PCA. They are defined by the rules that tell which elements to consider in specific circum-stances and consequently which path to take. As depicted inFig. 4, rules apply directly to the links that bind elements of the method (see the element Condition). As we will see later, these represent just one category of rules that we deal with in the PCA.

Fig. 5shows an example of a part of a base method that has been developed during the evaluation of the PCA in one of the participating companies. It represents how the analysis is performed in that company. More details on the base method content can be found in Section5.4. The reason the illustration is provided here is to show how pro-cess ﬂow and structure rules constrain the links between method elements.

Besides the rules that put constraints on the links between elements of the method there are also other types of rules that play important role in the PCA. In general, they can be categorised intoconstraint rulesandfacts. Since in performing the PCA rules play essential role we will explain their taxonomy in more detail.

4.4.1. Constraint rules

Constraint rules can be seen as assertions that constrain some aspect of the PCA for constructing project-speciﬁc methods. They can be decomposed into four subgroups:

process ﬂow rules, structure rules, completeness rules and

consistency rules.

Process ﬂow rules are rules that deﬁne conditional tran-sitions among activities in the process view of a method.

(10)

They deﬁne the conditions that have to be met to perform a particular transition. For example, in Fig. 5, the rule R1

deﬁnes a conditional transition to the activity Analyse Logical Structure while the rule R2 determines in what

circumstances the activity Analyse Logical Structure can be omitted.

Similar to process ﬂow rules are rules that belong to the structure rule category. Their distinction is that they can constrain any link between method elements and not just links between activities. In Fig. 5, the rule R4 represents

an example of a structure rule. It constrains the link between the activity Develop Prototype of the System and the toolDelphi.

Since both process flow rules and structure rules con-strain links between method elements they can be specified by providing the link to which they apply and the condi-tions that they require to be satisfied. Please be aware that such links do not necessarily connect content elements directly (e.g. two activities) but can also connect content elements with process flow elements, such as join or syn-chronisation (e.g. a link between an activity and process flow element or vice versa) or even two process flow ele-ments (e.g. a link between a decision node and synchroni-sation point).

Structure and process flow rules that belong to a base method of a particular organisation actually define project characteristics that are important at a particular stage of any project performed by the organisation. Their condi-tional part can be specified using the following template: [C1LO C2,. . .,LOCn] whereLOis a logical operator andCi

an atomic piece of the condition.Cicorresponds to the

fol-lowing form: [ProjectCharacteristicAO Value] where AO is

an arithmetic operator and ProjectCharacteristica named characteristic of the project.

Examples of process ﬂow rules (rulesR1,R2andR3) and

structure rules (ruleR4andR5) are provided below.4 •R1: if the process is in the decision node 1 and thescope

of the system is largeorincremental SDLC is chosenthen go to activityAnalyse logical structureof the system.

•R2: if the process is in decision node 1and thescope of

the system is not largeandincremental SDLC is not cho-senthen go tosynchronisation point 2.

•R3: if the process is in decision node 2 and problem

domain is new orcustomer requires the prototype of the system then go to activity Develop prototype of the system.

•R4: if the process is in activityDevelop prototype of the

system and the time frame for producing the prototype is more then 1 monththen develop the prototype of the system using Delphi tool.

•R5: if the process is in activityDevelop prototype of the

system and important reports are to be developed

then create output artifact Reports as a part of the prototype.

The rules above are written in natural language. Their conditional parts are underlined. If we used the template [C1 LO C2,. . .,LO Cn] the rule R1, for example, would be

rewritten as follows: [scope of the system= large OR

SDLC = incremental].5

Project characteristics, such as project length, project risk,project complexity,the scope of the system,the number of parties involved, etc. and their respective domains are defined within the organisation’s base method. However the values that these characteristics receive are project-spe-cific and are thus defined during the PCA execution. As you might noticed, assertions can be very concrete, such as in the ruleR5that asks whether important reports are

to be developed.

Besides process ﬂow rules and structure rules that both put constraints on associations between elements of a base method the constraint rule category comprises also com-pleteness and consistency rules. The purpose of these two subcategories is to assure that each project-speciﬁc method, created from the elements of a base method, is complete and consistent.

Completeness rules apply – in contrast to the process flow rules and structure rules – to a metamodel and not to a base method. Their responsibility is to define the con-ditions that must be met when creating a project-specific method. Completeness rules actually help to check whether a project-specific method that has been created includes all required components. For example, an organisation may decide the following rules have to be followed when creat-ing methods for projects:

•R6: each activity except the last onemust have at least

one successor activity.

•R7: each activity must be linked with exactly one

role.

•R8: each techniquemust be linked with at least onetool,

etc.

Completeness rules can be speciﬁed in a simple manner using attributes in the metalink class (see Fig. 4). Two attributes are suﬃcient: one stating the minimal and other the maximal number of connections between the metaele-ments a particular metalink is connecting. For example, the rule R7 applies to the metalink between the

metaele-ments activity and role specifying that the minimal num-ber of connections is zero and the maximal numnum-ber of

4 _{The rules are here written in natural language to ensure their}

understanding. When used in the PCA they need to be translated into the syntax dictated by the supporting tool.

5 _{While it may seem very time consuming to specify process ﬂow and}

structure rules in such a manner, this could be done easily if the tool supports graphical process modelling. In the toolset that we developed in support of the PCA the modelling feature was included and the speciﬁcation of process ﬂow and structure rules was found rather simple.

(11)

connections is one. Note that if the rule put a constraint on the number of activities per role, it would apply to a diﬀerent metalink.

Consistency rules are the last category in the group of constraints. They are similar to completeness rules. Their goal is to assure that the selection of fragments compris-ing a project-speciﬁc method is consistent. While com-pleteness rules only apply to elements that are linked together, consistency rules deal with interdependency between any two elements. In other words, for each ele-ment e they determine a set of other elements E that need to be included into a project-speciﬁc method if e

is included. In the example below the ruleR9asserts that

the deliverable Business model is dependent on the activ-ity Business modelling. This means that if the deliverable

Business model is selected for the inclusion into a pro-ject-speciﬁc method, the activity Business modelling has to be selected too. While such a dependency may seem trivial it is important as it helps to avoid conﬂicting sit-uations. Let us consider the situation inFig. 6. The rules

Ra and Rb are both structure rules. Since they deﬁne

project circumstances independently from each other it might happen that they are in conﬂict. For example, the rule Ra discourages the production of the Business

model artefact while the rule Rb requires it. With the

consistency rule R9 such a conﬂicting situation would

be prevented.

• R9: the deliverableBusiness Modeldepends on the

activ-ityBusiness modelling.

Consistency rules can be speciﬁed using the following template: [IFeTHEN E], where eis an element andE a set of elements thatedepends on.

The completeness and consistency have been identified already by pioneers of the method engineering field as the two most important criteria for internal or situation-al-independent quality of a situational method (see [7,18], and [14]). Since in the PCA, situational methods are cre-ated by adjusting an existing method (base method), for which we have to take care before that it is complete and consistent, these two qualities should not be too difficult to achieve. The possibility that we end up with an incon-sistent or incomplete method should be actually much lower as we do not start from a collection of fragments

that can be combined together in whatever way. Instead, we start from a base method that presents a collection of fragments that are bind together by constraint rules which put limitations on what we can omit from the base method. If captured properly, a base method should con-sist of several concon-sistent and complete subsets of frag-ments that correspond to particular development situations.

4.4.2. Facts

Another important group of rules that are considered during the PCA are facts. Facts are assertions that define characteristics of the project for which we create a pro-ject-specific method. Depending on how they define project characteristics they can be classified into base facts or derived facts. Base facts define project variables directly while derived facts are derived from base facts using infer-ences or calculations. In the examples below, the ruleR10is

a base fact while the rule R11is a derived fact. •R10: the project domain is well known.

•R11: if the project ﬁeld is telecommunications or

health-care then the project domain is well known.

In the PCA facts are very important as they are checked when structure and process flow rules are processed. For example, a structure rule might state that ‘‘when perform-ing requirements validation there is no need to produce a prototype if the problem domain is well known’’. To be able to perform this rule we must first check the facts about the project domain to find out whether the domain is well-known.

As indicated in the examples of the constraint rule cate-gory (see e.g. rulesR3orR5) facts can describe virtually any

condition that is important for the project. Furthermore, they are created dynamically during the PCA performance. For example, when an elementais selected to be included into a project-speciﬁc method this becomes a fact (a is selected) which could become important latter on in the PCA performance.

The metamodel in Fig. 7 shows the relationships between diﬀerent categories of rules and their components that play important role in performing the PCA.

4.5. The PCA algorithms

The algorithm that supports the PCA is relatively simple. It starts with an element in the base method (typically this would be a starting activity) and ends when there is no link that would connect the current element further with any other element. If such links are found they are examined for constraints they might have. When a particular link has no constraints or when constraints exist but are satisﬁed than the element at the end of that link is processed in the same way using recursion.

(12)

PROCEDURE CreateProjectMethod (pm, e);

//pm – project method, e – starting element of the base method

BEGIN

Find links for the element e For each Link l

IF conditions are satisfied for the link l THEN

Mark the output element of the Link l as selected for the pm

Mark the link l as selected for the pm CreateProjectMethod (output ele-ment of the Link l, pm)// recursion END IF

NEXT END;

When a project-speciﬁc method is created using the algorithm above, the elements that have been selected has to be checked for consistency and completeness. The veri-ﬁcation algorithms below show how this can be done.

PROCEDURE CheckCompletness (pm);

//pm – project method BEGIN

//completeness verification Select all links from the pm For each Link l

//Check the completeness constraint for the link l

Count the links that connect the input element of the Link l with the output elements of the same type as is the out-put element of the Link l

IF the number of links is outside the min, max limits then mark the Link l as problematical.

END; END;

PROCEDURE CheckConsistency (pm, e);

//pm – project method, e – starting element or

//link of the project-specific method BEGIN

//consistency verification

Select the set of elements and links D that e is dependent on

For each element or link d from D

IF d is not selected THEN Mark d as problematical

CheckConsistency(pm, d) //recursion.6

END; END;

To conclude this section it is worth to mention that as a part of the MasterProc project we also developed a tool that supports all the activities of the method reengineering framework, including the activities related to the PCA approach. Since the detailed description of the tool would be out of the scope for this paper, let us just emphasise that in method engineering an appropriate tool-support is indis-pensable. For the interested reader, a thorough discussion on the existing tools in the method engineering ﬁeld can be found in[23].

5. Evaluation of the PCA

5.1. Background

As described in Section 2, the PCA has been developed as a part of the MasterProc project. The ultimate goal of the MasterProc was to improve software development practice in Slovenian companies. To reach this goal our idea was to facilitate the companies with a framework and tool-support for reengineering their ways of working, i.e. producing software, so that the gap between their offi-cial methods (documented methods they claim to follow) and the ways how they actually develop software (methods in-action) would be as small as possible. The PCA presents an important part in this framework, as it suggests how to incorporate flexibility into formalised or documented methods, so that they could be adjusted to suite best to cir-cumstances of a particular project. Furthermore, the PCA tells how to describe the ways of working in an organisa-tion (organisaorganisa-tion’s base method) so that project-specific methods could be than created automatically by using appropriate tool-support.

In this section we describe how the evaluation of the ﬁrst version of the PCA has been performed. Please note that the MasterProc is ongoing project and that the next cycle, in which we expect to receive more valuable feedback from the participating companies is yet to come.

5.2. Evaluation criteria

Evaluation of a designed IT artifact requires the defini-tion of appropriate metrics and possibly the gathering and analysis of appropriate data [17]. In our case, the evalua-tion was performed by testing the PCA in laboratory envi-ronment with artificial data, and by observing the use of the PCA in real settings provided by the participating com-panies. The evaluation in laboratory environment was restricted to functional testing, e.g. to verify how useful is the tool-support for designing and capturing base methods and to create project-specific methods. More important however was the evaluation performed in the participating companies which we describe in this subsection. The aim of this evaluation was to monitor the use of the PCA in multi-ple projects and to collect data that could show how useful is the PCA in real environment.

6 _{To prevent the algorithm to come into a dead lock, there should be no}

(13)

When selecting quality measures for the evaluation of the PCA, our intention was twofold: ﬁrst to conﬁrm that the assumptions on which the PCA was based are correct, and second, to show that comparing to traditional assem-bly-based method engineering approaches which are criti-cised in the literature for their complexity, the PCA introduces an approach that is simple enough to be absorbed by the software development companies.

One of the characteristics that distinguish the PCA from other approaches to method engineering is that in the PCA, project-speciﬁc methods are created from the organisa-tion’s base method which documents the ways of working in that organisation. We believe that in each software development organisation, patterns of work could be found that tell how the organisation is developing software. While a large percentage of software companies own some kind of formalised method (typically commercial methods), empirical investigations show that what they really do on

IT projects differs a lot from what is written in the methods they own (e.g.[10,3]). Our assumption in the PCA is that in a typical software organisation the ways of working are sufficiently repeatable to be captured into a formalised method (base method) that would reflect how the organisa-tion actually performs its IT projects. The quality measure that could prove this assumption is the number of changes that we have to do to the base method each time when we want to create a project-specific method. We expect that along with the evolution of the base method this number gradually reduces till it finally reaches the point when it becomes insignificant (see also Section4.3).

Another assumption that was made during the design of the PCA was that by adjusting the base method of an orga-nisation it is possible to create project-speciﬁc methods which closely match the methods that are then really used in action. The quality measure that was used to validate this assumption was based on the number of changes that

(14)

were required during the project to make the project-specif-ic method – created from the base method – consistent with the real ways of working on the project.

More challenging in the evaluation of the PCA is how to show that the approach is powerful but yet easy enough to be absorbed by software development companies. The quality attribute that would clearly indicate this is the num-ber of companies that successfully implemented the approach in their practice. Since the PCA as described in this paper presents only the ﬁrst version of the approach for which the most important feedback has not been yet received we left this part of the evaluation to the next peri-od of the MasterProc project. It has to be mentioned how-ever that all the companies that participated in the ﬁrst learning cycle of the project (see Section2) agree to remain involved which could be seen as an indication of their belief in the usefulness of the approach.

Another issue that we wish to stress at this point is that the measurements that we describe in this section were all con-ducted in controlled environment, i.e. with the researchers that helped the companies with the creation of the base meth-ods, the usage of the tool, etc. This is important to notice, as it probably had some inﬂuence on how the developers and methodologists really acted on the projects.

In the further text we provide the results that were obtained by evaluating the PCA in one of the participating companies.

5.3. The company’s proﬁle

The company that we selected for the ﬁrst test is a typ-ical software developing organisation. They develop soft-ware based on an object-oriented approach by using their own application framework. It employs over 50 developers and typically takes middle-sized projects that require teams from 3 to 10 developers. A majority of the company’s

developers is well experienced and skilled. A small group of the company’s programmers is dedicated to maintain legacy systems that are still in production. This group works in older development environments and follow tra-ditional structural approach. They are less experienced in using modern information technologies. Before the pilot project was undertaken the company did not have any formal method as such but was developing software based on purely informal agreements and undocumented rules.

5.4. Developing a base method

Our first step in the pilot project was to develop a base method for the company using the approach that we briefly described in Section4.3. By inspecting the existing ways of working in the company we identified the elements that were technically and socially sound and those that were somewhat problematical. Detailed information on measur-ing socio-technical suitability of a method can be found in

[3]. Close examination of the existing practice revealed that there were some similarities between their approach and the method RUP (Rational Uniﬁed Process) for which they had been trained before. Based on the conducted analysis several improvements to the existing practice were suggest-ed and discusssuggest-ed with the company’s development team. The main objective of the evaluation at this stage was how-ever to conﬁrm the correctness of the assumptions on which the PCA was designed. To this end, we left the improvements of the existing practice to the next stage of the project which is dedicated to implement the improve-ments in an appropriate way (seeTable 1, Goal 3).

Our next step was to design a metamodel that would underpin the company’s base method. Fig. 8 depicts the metamodel elements and relationships that were chosen to adequately represent the company’s method. They rep-resent those parts of the method that the company wanted

(15)

to have in documented form. In the core of the metamodel is the element activity which is in relationship with many other elements. For example, each activity has arole that is responsible for its execution. It is supported byguidelines

and can produce or supplement many deliverables. Each deliverable can have a predeﬁned technique which is used to produce the deliverable. Techniques are further supported bytools. Bothtemplatesandexamplesof delive-rables could be provided to help to produce them. Activi-ties are further grouped intodisciplines.

Besides the aforementioned elements which all are of the content type (see alsoFig. 5) the metamodel includes ele-ments for supporting the flow of activities. The recursive relationship of the element activity supports simple transi-tions between activities while for more complex transitransi-tions specific control flow elements are required. As depicted in

Fig. 8, the abstract elementprocess ﬂow elementhas three subtypes: join, synchronisation and decision node, which support complex transitions. Note also that cardinalities of the relationships reﬂect the company’s completeness rules (see Section 4.4) and not associations in general. For instance, in general each technique can be supported by many tools but it is the rule of the company that there should be no more than one tool for each technique.

Once the metamodel was approved by the company, the formalisation started. The number of method fragments and rules that were initially captured during the formalisa-tion period is shown inTables 2 and 3, respectively. Note that base facts were not captured during the formalisation of the base method, as they belong to the deﬁnition of the project environment.

5.5. Creating project-speciﬁc methods

After the formalisation the base method was tested in three real projects. In this subsection we will describe how project-speciﬁc methods were created from the base method.

The projects that were conducted during the test were similar in technical terms and did not deviate much from typical projects the company had taken before. In all three cases the objective was to develop a web based, multi-user system with fat client in java. The company successfully reused its application framework and from this point of view there was no special action that had to be taken dur-ing the projects. The projects however diﬀered in size, scope and problem domain. Furthermore, they had diﬀerent development teams that diverged in the number and level of knowledge their members had about the problem domain.

For each of the projects a specific method was created taking into account the project characteristics. This was done in three steps: firstly the method was created from the company’s base method by using the PCA algorithm (see Section 4.5). The result was then verified manually by method engineers and where found incomplete or inap-propriate to the project specifics the base method was changed appropriately. The refinement typically included setting the status of selected activities to optional or decomposing deliverables into more detailed artefacts. Accordingly, several rules were changed or newly cap-tured. When the refinement was finally complete the PCA algorithm was restarted. The third step of the method creation was then performed during the project execution. If it was found that some fragment was miss-ing, redundant or inappropriate to the project require-ments the necessary changes were made immediately. Finally, when the project was finished, the method engi-neer took care that the same changes were replicated in the base method.

Table 4 shows diﬀerent data measured during the pro-jects. These are:

•Number of fragments in the base method: it tells the num-ber of fragments that were available in the base method before the initial conﬁguration was made.

•Number of initially selected fragments: it tells how many fragments were initially selected by the PCA algorithm by taking into account the projects speciﬁcs. We named this version of methodology initial methodology.

•Number of fragments additionally captured during the reﬁnement: this number tells how many fragments were additionally captured within the reﬁnement of the base method.

•Number of fragments selected after the refinement: after the refinement the PCA algorithm was performed again. This number tells how many fragments were selected for inclusion into the project specific method (refined method).

Table 2

The number of captured fragments

Metaelement Number of fragments

Discipline 10 Activity 61 Role 24 Deliverable 52 Technique 14 Guideline 34 Tool 20 Template 8 Example 8 Sum 221 Table 3

The number of captured rules

Rule type Number of instances

Structure rules 54

Process ﬂow rules 29

Derived facts 27

Completeness rules 22

Consistency rules 55

Base facts 0

(16)

•Degree of consistency between initial and refined method: the number tells how coherent were the initial and refined method. The degree was calculated using formu-la(1), whereNrefinedis the number of all fragments in the

reﬁned method, Ninitial the number of all fragments in

the initial method, andNchangedthe number of fragments

in the reﬁned method not included in the initial method.

Drefined¼

NrefinedNchanged Ninitial

ð1Þ •Number of changes made during the project: during the project additional changes were made if the method was found inappropriate. This number tells how many changes were required so that the method was consistent with the real ways of working on the project. The result-ed method was namresult-ed ﬁnal method.

•Degree of consistency between refined and final method: similarly as above, this number tells how coherent were the refined and final method. The degree was calculated using formula(2), whereNfinalis the number of all

frag-ments in the ﬁnal method, Nreﬁned the number of all

fragments in the reﬁned method andNchangedthe

num-ber of fragments that were changed or newly captured during the project execution.

Dfinal¼1 Nchanged

Nrefined

ð2Þ

5.6. Discussion

It can be noticed fromTable 4that the number of frag-ments in the company’s base method significantly increased after its application in the three projects. The increase was particularly considerable after the first pro-ject. The explanation for this lies in the fact that the base method, which was initially captured, was not fragmented enough to serve the PCA. Within the refinement the base method was then improved by adjusting the activities (several activities were added or decomposed into smaller units) and by the decomposition of the deliverables (since the first version of the base method included several com-posite deliverables, their decomposition contributed a lot to the number of fragments.). This has resulted in higher granularity of the fragments, which by nature brings more options for adaptability. In the next two projects the

increase in the number of fragments was not that signifi-cant. This can be supported also by considering the degree of consistency between refined and initial method. The numbers in Table 4, row 6, clearly show that the degree of consistency improved in each project. Similar finding is suggested by the degree of consistency between refined and final method (see Table 4, row 8).

As emphasised before, the projects that were conducted during the test were similar in terms of technology that was used to develop the solution. Furthermore, in all three cases object-oriented approach was followed. We antici-pate that for this type of projects the learning phase of the base method is nearly finished and that in its future use it will not be necessary to make any major modifica-tions. If taking different projects however, e.g. following structural approach, using different kinds of technologies (without reusing the application framework) etc. the base method would have to go through similar steps.

We are aware that the results presented in this section can not add much to the validity of the PCA, as they do not address quality attributes that would imply the PCA is useful in practice or better than existing approaches. The evaluation that will address these issues is yet to come. Our intention at this point was to show that the assump-tions on which the PCA is designed and which are rather unique (the PCA is the only approach to method engineer-ing that creates project-speciﬁc methods from the method that elaborates patterns of work in a particular organisa-tion) are correct. We believe the results are important for the initial iteration of the MasterProc project and that they suggest the project should continue.

6. Comparing and combining the PCA with other SME approaches

The approach that we described in this paper builds on the previous research work on the method engineering ﬁeld. Its main contribution should be recognised in its abil-ity to execute. In this section we will ﬁrst describe some of the features that distinguish the PCA from other SME approaches. It is important to stress however that our ambition is more than just to make another method for SME. We believe the PCA can act as a complementary approach and together with other SME approaches form powerful combination. A brief discussion on how the

Table 4

Data collected during the projects

Project 1 Project 2 Project 3

Number of fragments in the base method 221 281 296

Number of initially selected fragments (initial method) 102 82 91 Number of fragments additionally captured during the refinement 56 14 2 Number of fragments selected after the refinement (refined method) 92 86 90 Degree of consistency between initial and refined method (Drefined) 0, 655 0, 848 0, 920

Number of changes made during the project (final method) 11 5 2 Degree of consistency between refined and final method (Dfinal) 0, 880 0, 942 0, 978