Analysis of Agile Software Development from the
Knowledge Transformation Perspective
Ilia Bider
Department of Computer and Systems Sciences (DSV), Stockholm University (SU)/ IbisSoft AB, Stockholm, Sweden
ilia@{dsv.su.se|ibissoft.se}
Abstract. While the Agile Software Development (ASD) has been successfully pro-moted in the last 15 years, there is no agreement on how to determine whether a par-ticular project is agile or not. Some practitioners consider agility as strict usage of a specific methodology, e.g. SCRUM, others consider agility as adhering to Agile Manifesto. The lack of common view on ASD prevents creating common guidelines on when the usage of ASD is appropriate. This paper presents a model of ASD that helps to differentiate it from the traditional, phase-based development, and more strictly defines the area of its applicability. The model has been built based on the knowledge transformation perspective, as the author considers it to be the most dif-ferentiating perspective when comparing ASD to traditional software development. For building the model, the ideas from SECI model of Nonaka have been exploited. The results, in the form of requirements to be fulfilled for successful employment of ASD, are demonstrated through analysis of completed ASD projects.
Keywords: Software Engineering, system, software project, agile, knowledge
trans-formation, SECI model
1
Introduction
Agile Software Development (ASD) has appeared as a reaction on the increasing rate of changes in system requirements, see, for example, [1]: “requirements change at rates that swamp traditional methods”. Though there are numerous books and articles on ASD, e.g. [2], there is no complete agreement on the definition of what ASD is. The main problem here is that the essence of ASD is sometimes mixed with particular project methodologies, like SCRUM, that could be used in both agile and non-agile software projects.
The existence of the confusion of what agile means could be easily seen when investi-gating the online discussions devoted to the topic, such as “Do you agree or disagree that Scrum is not Agile?” [3]. This corresponds to the authors own experience of investigating software projects referred to as agile because they use SCRUM, despite the fact that they concern the development of hardware-near software with for each release fixed require-ments specifications [4].
Currently, the only definition of ASD on which there is a common agreement, is the so-called agile manifesto [5]. The weak side of the manifesto is that it does not have an
under-lying scientific theory, but consists of a number of principles that allows different interpre-tations. For example, [6,7] consider the lack of theory and philosophy behind Agile Mani-festo as a root of misunderstanding of agility not only in software development, but also in other related disciplines, e.g., Information Systems (IS). Their consideration is a result of reviewing agility in many disciplines. They also claim that the lack of considering agility outside the software development makes this manifesto not generally accepted by the wider audience.
The goal of this paper is twofold, namely, it is aimed to suggest a theoretical framework that could help in (a) explaining the difference between the agile and non-agile software development; (b) defining the area of applicability for ASD. In this paper, we are not fo-cused at discovering new facts about ASD, but rather on finding a practically useful framework to explain the existing facts, so that they can be understood by all stakeholders concerned, including management of software vendors and customers.
To attain the goal above, we will build a dynamic model of a software project that em-ploys the ASD methodology and compare it with the same kind of model for traditional, phase based software development. Based on this model, and comparison with traditional development, we will derive the requirements on the software project in which ASD could be successfully employed.
When building a model for the purpose of analysis of agile development, we took a knowledge transformation perspective on software development. When developing this perspective, we used the ideas of Nonaka [8] on knowledge transformation in organiza-tions. In particular, our model was influenced by Nonaka’s SECI model, where SECI stays for Socialization – Externalization – Combination – Internalization. The SECI model ex-plains how the knowledge is transferred/and or converted inside and between two different forms – tacit (in the heads of people) and explicit (in the form of writing, drawings, math-ematical models, etc.).
The paper is structured in the following way. In Section 2, we give a short overview of our method and background on which we built our framework. In Section 3, we present an informal description of the proposed framework. In Section 4, we use this framework for creating requirements on the project that need to be fulfilled in order for the agile approach to be successfully employed. In Section 5, we apply the results from Section 4 to analysis of past software projects from the literature and from the author’s own practice. In Section 6, we discuss the result achieved so far, and outline plans for the future.
2
Method and background
In our research, we employ a Design Science Research (DSR) approach, more specifically its interpretation according to [9]. In this interpretation, DSR, as a way of generating and testing hypotheses for generic solutions, requires researchers to act in two different worlds: (a) the real world of specific problems and solutions in local practices, and (b) the abstract world of generic situation, problems and solutions. There is no specific require-ment on the order in which the moverequire-ment is completed. Researchers can start with search-ing a solution for a known in a local practice problem before or after generalizsearch-ing it, or they can start with building a solution for the problem unknown, and then finding what
the solution is good for. The main point is to have in the end a description of the triad <generic-situation, generic problem, generic solution> and one or several test cases that shows that the generic solution applied in a specific situation can solve the problem.
In our case, we are identifying conditions for success when the agile approach to Soft-ware Engineering (SE) is employed in the project. These conditions can be used for solving a number of problems that exist in SE practice that are discussed later in Section 4. In sec-tion 5, we will present test cases of using the solusec-tion developed for analysis of completed projects.
Agile system development is a concept that includes many variations, like eXtreme pro-gramming, test-based development, peer propro-gramming, etc. To answer the question of agile applicability in general terms, we cannot rely on building a model based on a particular variation/interpretation of Agile Software Development (ASD). The model we need should be on a higher abstract level that includes no details, but all essential features of the agile approach. Our model of ASD and explanation of how it differs from the traditional one was built based on the following four sources:
1. Own experience of software development, agile as well as not agile, in various capaci-ties, e.g., as a programmer and a group leader in larger projects, and as a project leader and architect in smaller R&D projects, see for example, [10].
2. SECI model of Nonaka [8] on knowledge transformation, which gave us the idea to build a model based on knowledge transformation perspective. More exactly to consid-er software development cycle as knowledge transformation cycle.
3. Good regulator theorem of Conant and Ashby, which gave us the idea to introduce a new type of knowledge absent from SECI model – embedded knowledge.
4. Agile Manifesto [5]. We have chosen Agile Manifesto from the vast literature on ASD based on the following reasons: (a) it comprises the essence of ASD without going into details of specific methods; (b) as far as we know, this is the only document on which the community of practice agrees, therefore it can be considered as a collective wisdom of the community.
We do not claim that the sources above are the only ones that could be used for answering the question posed in the title. However, they were sufficient for us to design the solution we were looking for. Other sources, when discovered can be used for validating and refin-ing the solution, some of them are mentioned in Section 6.
As sources 2 and 3 are less known in the domain of SE, below, we present a short over-view of them. The SECI model, where SECI stays for Socialization – Externalization – Combination – Internalization, by Nonaka [8] explains the ways of how knowledge is cre-ated in an organization while being transformed from the tacit form (in the heads of the people) to the explicit one (e.g. on the paper in the form of texts, drawings, etc.) and back, see Fig. 1. [8] defines the cycle of knowledge creation as consisting of four steps or phases: 1. The cycle starts with Socialization (left top corner of Fig. 1), where tacit knowledge is transferred from the heads of one group of people to others via informal means, con-versations during the coffee breaks, meetings, observations, working together, etc. 2. The next phase is Externalization, which is the conversion of knowledge from the tacit
3. The third phase is Combination, which is transforming the externalized (explicit) knowledge in a new form using existing knowledge, e.g. solution design principles (right bottom corner of Fig. 1).
4. The last phase is Internalization, which converting the explicit knowledge, e.g. a solu-tion, in the tacit knowledge of people that are ready to apply this knowledge to any sit-uation that warrants it (left bottom corner of Fig. 1).
The cycle of SECI can be repeated indefinitely reflecting constant creation of new knowledge.
Fig. 1. SECI diagram, adapted from [8]
The SECI model identifies two type of knowledge tacit and explicit. This is not enough if we want to apply knowledge transformation perspective to software development. We need to consider the knowledge built in the software system, creating which is the goal of any new software development project. This knowledge cannot be regarded as explicit as people using the system may not know how it works internally. This knowledge cannot be called tacit as it belongs to the system not to the human beings (unless we consider the system being an agent comparable with the human being). We refer to the knowledge built-in built-into the system as to embedded knowledge. The existence of embedded knowledge is indirectly supported by Conant and Ashby theorem [11] that “every good regulator of a system must be a model of that system”. Using analogy, we can reformulate (specialize) the Conant and Ashby theorem as:
“Every good software system is a model of the requirements its implements/satisfy” The latter is indirectly conformed by the practice of reengineering of legacy systems for which the source code has been lost or is so dirty that it requires too much time to under-stand the details. It also is in line with the author’s own experience of supporting and fur-ther developing legacy system built by unknown people.
Adding an additional category of knowledge, we applied the ideas from [8] to the cycle of software development both in its traditional and agile forms, and built models that repre-sent these methodologies of software development. In this paper, we limit ourselves to discussing full cycle of new product (software) development, leaving the issues of
applica-bility of the agile methodology to maintenance of the existing products built in the tradi-tional way outside the scope of this paper.
When considering requirements on software project for successful employment of ASD, we take a system perspective on the software project and its environment. More exactly, we consider that there are three distinct but interconnected systems involved in any soft-ware project:
1. The software system (S-system), i.e., the virtual artifact being developed or modified. 2. The software project (P-system), i.e., the work system undertaking the development or
modification of S-system.
3. The software context (C-system), i.e., the environment in which the software product is being, or is intended to be used.
The three-system division introduced above relates to many existing studies and frame-works. For example, [12] distinguished between the Narrow System of Interest (S) and Wider System of Interest (P), and argued that both exist in an Environment (C) and even Wider Environment. We are not going to discuss the system view any farther, but will use the division into S, P, and C systems when discussing requirements, as different require-ments will concern different systems, or interconnection between different systems.
3
Building models of software development
3.1 A model of traditional phase based software development
Applying Nonaka’s ideas to knowledge transformation during the traditional cycle of software development, we have got a model presented in Fig. 2. The model is abbreviated to ECEA, which stays for Externalization-Combination-Embedment-Adoption. The cycle starts with tacit knowledge on problems and needs in the heads of the stakeholders (hu-man participants of C-system ) and consists of four phases:
Phase 1 – Requirements engineering (RE) - consists of transforming tacit knowledge possessed by C-system participants into explicit knowledge of requirements (right top cor-ner of Fig. 3). This phase corresponds to Externalization from SECI model. The conversion is usually done via facilitating workshops where specialists in requirements engineering work together with the stakeholders and, possibly specialists in the domain of C-system. The stakeholders and domain specialists have tacit knowledge on the C-system with its problems and needs, while the RE specialists help to convert this knowledge into a struc-tured description in form of requirements.
Phase 2 – Design – consists of transforming explicit knowledge of requirements to ex-plicit knowledge software system design (right bottom corner of Fig. 2). This phase corre-sponds to Combination of SECI model. The conversion is done by applying contemporary or innovative design principles to the requirements in order to produce design documents, e.g. class diagrams, user cases which can be at the next phase converted to a software sys-tem.
Phase 3 – Coding consists of transforming the explicit knowledge of system design to the knowledge embedded in the system (left bottom corner in Fig. 2). Though parallel to
Internalization from SECI model, this phase does not correspond to the latter exactly. We call this transformation Embedment. For programmers engaged in this activity, this trans-formation is from one form of explicit knowledge to another (program code). However, for the C-system, the knowledge built into the S-system is not explicit. The conversion is done using contemporary or innovative methods of programming, e.g. object-oriented program-ming.
Phase 4 – Learning to use in own practice – consists of transforming embedded in the system knowledge into the tacit knowledge of how to use it (left top corner in Fig. 2) in C-system. Though parallel to Socialization from Fig. 1, this phase does not correspond to the latter exactly. We call this transformation Adoption. If the system is aimed to work in symbiosis with people “inhabiting” C-system, adoption includes the users learn how to use the system when completing their tasks. In case of pure technical systems, adoption in-cludes adjusting other component in C so that they can successfully work together. Nor-mally, both types of adoption happen for any S-system. Even when the system is purely technical, some human participants of C-system need to have tacit knowledge on the rela-tionships between the new S-system with other components of the context system. The latter is important in case a problem in C related to S arises.
Fig. 2. ECEA - Knowledge transformation in the traditional process development.
Note that the cycle above is idealization. For example, parallel to Embedment and Adop-tion there might be addiAdop-tional knowledge transformaAdop-tions:
• Writing manuals, which is converting embedded knowledge into the explicit one, • Reading manuals, which is converting explicit knowledge in manuals into the tacit one However, these two transformations are losing its importance as the contemporary users learn new systems in the try and error fashion without bothering to read manuals.
As with the SECI model, the cycle of Fig. 2 can be repeated as soon as there are needs to change the software system. We also need to point out that the phases represented in Fig. 2 need not be executed in a strict sequential order as prescribed by the waterfall methodol-ogy. The first three phases can run in parallel and affect each other. For example, the RE may pass the main requirements to the designers without uncovering all requirements, so that the designers can analyze whether the requirements already discovered can be imple-mented in a software system, or some of them need to be renegotiated. The same kind of parallelism can be arranged between Design and Coding. This comment means that the model on Fig. 2 is not representing the waterfall model only, and it allows iterations and conversations between the phases, as it is done in so-called V-model. These details, how-ever, are not essential for the purpose of the modeling accepted in this paper.
Summarizing the deliberation above, as traditional we consider software development in which all four phases are separated, independently whether they are completed in se-quential (e.g., waterfall model) or non-sese-quential (e.g., V-model) order. This is different from the agile development as will be shown in the next subsection.
3.2 A model of agile software development
From own experience, as well as from literature on Agile System Development (ASD), see for example [1,13], the main difference between Traditional System Development (TSD) and ASD is that the latter relies much more on the tacit knowledge than on the explicit knowledge. In terms of the knowledge transformation cycle of Fig. 2, ASD tries to shortcut the phases of Externalization and Combination and go directly to Embedment.
Shortcutting Externalization and Combination in Fig. 2, we get a model in Fig. 3. The model is abbreviated to SEA, which stays for Socialization-Embedment-Adoption. The SEA cycle differs from ECEA cycle in three respects:
1. The nature of the first phase in Fig. 3 is changed against Fig. 2. It consists in transfer-ring tacit knowledge on the problem and needs from the stakeholders to the develop-ment team. This phase corresponds to Socialization in Fig. 1.
2. Design and coding are merged into one phase Embedment, which we continue to call Coding.
3. In addition, one big cycle is substituted by many smaller and shorter ones. The system is built iteratively starting with the basic functionality. During the exploitation of the basic system, better understanding of the needs is acquired, which is converted in add-ing details to the system in the next iterations. In other words, agile methodology is based on a simple motto “Develop and introduce in practice as little as possible as soon as possible, and build upon it in the following iterations”.
As with ECEA, the SEA model is an idealization. During both Socialization and Embed-ment, some knowledge is explicated, e.g., simple flowcharts drawn on the paper, or on the black/white board, or user stories are written as plain text. This, however, is done in order to facilitate the tacit knowledge transfer from one group of people to another, or into the system, not to make all knowledge explicit. Therefore, using explicit knowledge in Social-ization of Fig. 3 is different from its usage in ExternalSocial-ization of Fig. 2, where the whole
phase is directed at creating explicit requirements before going further with Combination and Embedment.
As with the traditional development of Fig. 2, having different phases in agile develop-ment does not mean that they need to be impledevelop-mented in sequence. Both Socialization (phase 1), and Embedment (phase 2) can be completed in parallel.
Fig. 3. SEA - Knowledge transformation in the agile software development
4
Analyzing the models
4.1 Advantages and drawbacks of traditional development
In this section, we will use the ECEA model to analyze strengths and weaknesses of the traditional approaches to software development. Base on phases identified in Fig. 2, we can identify the following advantages of the traditional development.
1. There are 4 distinct phases of knowledge transformation that allows distributing the work between the experts in four different areas: RE, System Design, Programming, and Adoption (including user training, writing manual and project management for sys-tem adoption). Using explicit form as an output of Externalization and Combination facilitate communication of the specialized experts in different areas.
2. Having explicit requirement specifications, and design specifications facilitates using existing knowledge on design principles and programming.
3. Having explicit requirement specifications allows entering a contract agreement with fixed obligations on the side of the development team.
The drawbacks of using traditional development are the other side of its advantages: 1. Instability. A small error or missing a requirement in Externalization may be amplified
ab-sent in Combination and Embedment there is little chance that a miss will be recovered at these phases. In addition, specialization may result in the experts in the design and programming having no knowledge, skills or experience to communicate with the stakeholders.
2. Fuzzy requirements. Using requirements as a basis for design, and a contract agreement presumes that all such requirements could be understood and formulated, and the cus-tomer has skills and experience of imagining how the C-system will work when an S-system that satisfies the requirements is put into operation. In situations when there is a reference point, for example, an older S-system already exists in the C-system, such a presumption could be warranted. However, when such reference point does not exist, for example, when the C-system should be significantly changed when a new S-system is deployed, such presumption is not warranted. In the latter case, insisting on fulfill-ment of the unchangeable requirefulfill-ments as a basis for the design and the contract could lead to a catastrophe, more exactly, the system satisfying all requirements, but being useless.
3. Evolving context. For a large complex S-system, it may take too much time to go through one big cycle. While a new S-system is being developed, the C-system can un-dergo changes, e.g., problems and needs may continue to evolve. As the result, the S-system could be outdated before it is deployed in practice of C-S-system. This could be-come a major obstacle for dynamic contexts.
4.2 Requirements on agile software development
Agile software development is a reaction on the drawbacks of the traditional one. Its main principles are directed toward mitigating the three problems (drawbacks) listed in Section 4.1, namely:
Socialization as the first phase of the agile cycle of Fig. 3 is aimed at mitigating the first drawback of STD (see Section 4.1). It ensures that the whole development team gets tacit knowledge on the problems and needs that they convert into the S-system at the Em-bedment phase. Running Socialization and EmEm-bedment in parallel allows the team to re-fine their understanding of problems and needs even after starting writing the code.
Focusing on producing software without producing requirements and design documen-tation is aimed to mitigate the second problem on the list of drawbacks of TSD from Sec-tion 4.1. DemonstraSec-tion of software can spin imaginaSec-tion and facilitate refining under-standing of the needs based on the possibilities to be provided by S-system.
Splitting one big cycle into a series of smaller ones is aimed at mitigating the third problem on the list. It allows putting into operation the first version of the S-system in a shorter time, and thus avoiding it being outdated for a dynamic C-system. Running Social-ization, and Embedment in parallel also contributes to the same goal as changes in C-system can be discovered and take care off on the later phases of the development. Split-ting the cycle also helps alleviaSplit-ting the second problem, as only actual introduction of an S-system into the context (C-system) can show how well these two systems fits each oth-er. A misfit of the first version is possible to fix in the next version.
However, while alleviating the problems of the traditional software development, agile model sets requirements on all three interconnected system P, S and C, thus limiting its
area of application. Below we list the requirements based on the analysis of SEA model of Fig. 3:
1. As there is no explicit requirement specification or design documents, the whole devel-opment team should be engaged in acquiring tacit knowledge from stakeholders. It means the needs of switching from specialization of traditional team to universality of the agile team. All members need to have knowledge on all aspects of software devel-opment, from business and system analysis to programming. It does not mean that all of them need to be equally proficient in all branches, but they need to have enough un-derstanding of all of them to be able to acquire and transform the knowledge through informal meetings without writing lengthy documents. This is a requirement on P-system - you cannot run an agile project if you do not have such a team or cannot cre-ate it in the first phase of the project.
2. Equally important is the attitude of the stakeholders inhabiting C-system. Firstly, they need to assign time to work with the P-system team during the whole project, not just in a short series of facilitating RE workshops. Equally important is that they have pa-tience when communicating with the developers who might not know their domain and can raise seemingly silly questions.
3. As there are no explicit requirements there is no standard document on which a con-tract between stakeholders and developers can be based. The relationships between them should be regulated having some other scheme in mind.
One of the main features of the agile software development is that instead of delivering an S-system in one step, the delivery is split in several cycles each of them resulting in deliv-ering an enhanced S-system for its implementation inside C-system. Each cycle should be finished in the shortest possible time to mitigate the problem that arises from the dynamic context. The following three requirements are connected to the cyclic delivery:
4. Providing the first delivery in a short time implies that a relatively small core S-system could be successfully implemented in the given C-system. Actually, this is a require-ment on C-system, which can be expressed as possibility to identify a small core S-system implementation of which has sense (is meaningful). This, obviously, is not al-ways true. For example, suppose a new S-system should substitute an older (legacy) system. It is difficult to imagine that a new core system that lacks some essential func-tions of the old one will be acceptable for such the substitution.
5. Splitting the delivery in a number of smaller cycles does not exclude, but rather pro-motes the final system becoming quite complex. Providing short delivery time for each iteration requires that enhancements added to each release do not require totally rewrit-ing the code of the core system or the code produced in previous iterations. Rewritrewrit-ing will increase time needed to produce any meaningful enhanced version of the system, and this time will increase with each iteration. Avoiding rewriting the code requires the core S-system having solid architecture that ensures the code remaining more or less the same during the whole project (lifetime of P-system). This is actually a requirement on S-system.
6. Short delivery time for each iteration requires high efficiency of coding which puts re-quirements on the languages and tools used for developing an S-system. It is not
prac-tical to use low-level languages; an agile project needs to employ high-level means that increase the productivity. These can be high-level domain specific languages, like 4th GL languages of 1980th and 1990th (that disappeared as class), libraries of functions or objects, development platforms, like Ruby-On-Rails, or executable models. These means could be acquired from a third party from the beginning of the project, or devel-oped in parallel with the S-system produced by the project. These means should be synchronized/aligned with the overall architecture of S-system. The requirement of employing high-level means can be considered as a requirement on P-system, and part-ly on S-system, the chosen means need to be aligned with the S-system architecture. The list of requirements above can be used for several purposes, including:
1. Analyzing what went wrong/right in a successful or unsuccessful attempt of employing agile methodology.
2. As a check list for decision whether employing agile methodology have chances for success. For example, if a customer cannot be convinced to ease the demand of having a contract based on the detailed requirements documents, employing agile methodolo-gy could be a risk factor. The same is true if there is no possibility to agree on a rela-tively small core S-system for the first iteration. Having a team with highly specialized members may also prevent agile development. In this case it might be preferable to use phase based scheme of Fig. 2, but run the phases in parallel.
3. As part of the plan of action when decision to use the agile approach has been made. For example, if agreement on the core S-system and agile methodology has been reached, one could hire a proper team, and choose proper means to boost productivity and ensure stable architecture.
4. The requirements above plus the models in Fig. 2 and 3 can be used as educational ma-terial that clearly shows the difference between the traditional and agile methodologies, weakness and strength of both, as well as their areas of applicability. Considering that the models are independent from a particular brand of traditional or agile methodology, such educational material has a potential to be widely used.
5
ANALYSIS OF SUCCESS AND FAILURES OF ASD
PROJECTS
To make the first verification of the suggested in Section 4.2 requirements, we analyzed two cases of completed ASD projects (item one on the list of the possible usage of re-quirements). The first case concerned an unsuccessful ASD project we found in the litera-ture. The second case was from our own practice; it concerned the development of the system described in [14]. Due to the lack of space, in this paper, we can only give a short overview of the results of the analysis of these two projects.
5.1 An attempt of substituting a legacy system in an agile manner
The story of this project was presented at Agile Conference in 2008 [15] under the name “When Working Software Is Not Enough: A Story of Project Failure”, and then repeated
at other events [16]. This project has been chosen for analysis due to the following rea-sons. Firstly, it is relatively well documented: besides a short presentation of the talk [15], the video record of the talk [17], and slides of the presentation [16] are available on the web. Secondly, this documentation contains analysis of the reasons for failure that can be compared with our own analysis. Thirdly, in addition to the analysis of the failure given by the project manager [15,17,16], an analysis by an independent expert is also available [18].
Based on the description [14,15], video recording [17], and independent analysis [18], we can conclude that the main reason for the project failure identified by the project man-ager corresponds to not fulfilling our requirements 2, and 3, i.e. stakeholders doing their part of job, and proper relationships between the people inhabiting C and P systems. As far as requirements 1 is concerned, the project manager considered the team being mature, and the materials analyzed does not provide enough evidence to make an independent judg-ment.
As far as requirements 4 is concerned, identifying a smaller core S-system that could be implemented in practice, the project failed to do so. They interpret the motto “Deliver working software frequently” only as demonstration, and testing by the customer (which has not been done as follows from the video recording [17]). The project could not identify and agree on a small core S-system to be directly introduced in practice. Though this fact is revealed in the presentation, especially in the video recording, the analysis did not consider failure to agree on a core system as a major reason for the project failure.
As far as requirements 5 and 6 are concerned, it seems that they were not fulfilled. The coding was done in C# and SQL, and changes in the requirements priorities caused a lot of code rewriting. The latter points out to lack of solid architecture. What is more, the analysis in [15,17] does not considers solid architecture and means of enhancing productivity to be prerequisites for the agile project success.
Summarizing the above, the analysis of this project based on the requirements from Sec-tion 4 confirms the analysis of the project manager as far as requirements 2 and 3 are con-cerned. In addition, it highlights the project not fulfilling the requirements 3 (identifying a smaller core S-system), and 5 and 6 - lack of solid architecture and means that enhance the efficiency. In our view, the last two contributed to the project failure in no less degree than not fulfilling other requirements.
5.2 Developing a system from scratch in an agile manner
The system called ePortal was developed for a large Swedish call center to solve the daily staffing problems that can be defined as follows. The system was developed by IbisSoft AB in the agile manner in tight collaboration with her customer Eniro AB, the author serving as a team leader of the IbisSoft’s team. The core system was successfully devel-oped and introduced in organizational practice in the spring of 2009. The project went through three major cycles of which the second was the most critical as it include adding 600 new users (agents) that could work from both home and the office. The system is fully operational and continues to be used without major problems.
Considering the requirement from Section 6, the project satisfied the requirements 1-6 in the following manner:
1. The development team consisted mostly of two members (the project leader including). Other specialists were called for solving or investigating specific technical problems. The team had several years’ experience of working together without detailed require-ments or design specifications using face-to-face communication as the main channel for communication.
2. The customer found it very convenient to work in a relatively informal manner around first a prototype, and then - emerging system. The customer was a geographically dis-tributed organization. Only IT function was in the same city as the development team, other stakeholders (including system users) were spread all over the country. Face-to-face meetings between the developers and stakeholders were rare. Communication was managed through (a) email, (b) phone, (c) the system under development that was ac-cessible for the stakeholders all the time. All testing was done by the customer, and the customer was responsible for implementing the system in their organization. The im-plementation was well planned, including the customer educational department creat-ing a video on how to use the system.
3. As the project did not rely on detailed requirements, the contract was based on the gen-eral estimation of the amount of work needed. Prioritization was done on the fly mov-ing some features to the future releases or adjustmov-ing the budget if some features were of great importance. The atmosphere of trust was created while discussing the prototype, and was strengthen after implementing the core system in organizational practice. 4. Agreement on the core system was reached very early in the project.
5. The architecture of the system was designed on both the logical and technical levels. 6. The technical architecture was tightly coupled to the tools we used for developing the
system, see below.
7. Our team was small, the same was true for the project budget. Even if we had desired using low-level programming, we couldn’t have afforded it. High-level means em-ployed in the project included Ruby-on-Rails (RoR) platform for developing the server side of the application, and Sencha’s ExtJS toolkit for developing the client side of the application.
As follows from the analysis above, using agile methodology was justified. Moreover, it led to successful completion of the project. Note also, that the project described above did not satisfy the commonly used recommendations such as having frequent face-to-face meetings with the customers. Still it satisfied the list from Section 4 and succeeded.
6
Contribution of our work in relation to works of others
This section is aimed at highlighting the contribution of our research while validating it through or contrasting it to the works of others. The main contribution of this paper is the models of TSD and ASD that highlight the difference between them and drawbacks and advantages of each, as well as the list of requirement on the P, S, and C systems that need to be fulfilled when ASD is employed. We also composed a list of practical tasks in which our results could be of use (see Section 4). We also made a test of usefulness of our sug-gestions in one of the practical tasks - analysis of completed projects (Section 5). This test uncovered some problems that were missed in analysis made by others.
Many of the assumption used in building the models, and conclusions derived from them are known and discussed in the literature, which we consider as an external validation for our work. For example, the weakness of the traditional development in the dynamic world is summarized in [1] as “requirements change at rates that swamp traditional meth-ods”. The reliance on tacit knowledge in agile development is well known, see, for exam-ple, analysis of literature related to the topic in [19]. The difference in usage of tacit and explicit knowledge between traditional and agile development is discussed in [13].
The fact that many of the conclusions we made based on our models can be found in various sources cannot be considered as weakness of our approach. Though an expert can identify whether a particular situation warrants a particular method based on his/her tacit knowledge, this is not true for the whole SE community, which is manifested in the high percentage of SE projects failures, including failing ASD projects (see Section 5.1).
The advantage of using our models for deriving conclusions is that while they are quite simple, they are powerful enough for the task at hands. In particular, they were suitable for deriving requirements on P, S and C systems that could improve chances for success of employing the agile methodology of software development. Even if we consider that all six requirements listed in Section 5 are not new, and can be found in various other sources, having them derived from analysis of two simple models has value, including a pedagogi-cal one. It may help to convince people to actually use the requirements and not rush to using agile software development just because it is in fashion right now. An additional advantage of these models is that they are generic and do not consider any particular brand of traditional or agile methodology. The decision makers do not need to understand the ideas behind Waterfall, V-model, XP, Scrum, etc. to understand the diagrams in Fig. 2 and 3, and what follows from them.
Our approach to building the models of TSD and ASD is based on the knowledge transformation perspective. Though this perspective is well known in management science [8], to the best of our knowledge, it was never fully exploited in connection to SE. For example, the only paper we found that considered software as knowledge embedment was [20]. The paper argues that software is not a product but a new kind of storage for knowledge of which it counts five types: DNA, Brains, Hardware, Books and Software. The usefulness of our models can serve as an inspiration to taking knowledge transfor-mation perspective in SE research more seriously.
Our future plans includes gathering more empirical data to support the assumption of usefulness of the ECEA and SEA models suggested in this paper, e.g. via case studies. Here, we are especially interested to see which type of tools agile teams use to enhance their productivity. Another topic worth investigation is how the knowledge obtained in one ASD project is transferred to other ASD projects considering the desire of not having any extended documentation. Our working hypothesis in this respect is that such knowledge is being built into the tools used in ASD.
REFERENCES
[1] J. Highsmith, K. Orr, and A. Cockburn. (2000) E-Business Application Delivery, pp. 4-17. [Online]. www.cutter.com/freestuff/ead0002.pdf
[2] J. Shore and S. Warden, The art of agile.: O’Reilly, 2008.
[3] Matthew Weaver. (2011) Do you agree or disagree that Scrum is not Agile? May. [Online]. http://www.linkedin.com/groups/Do-you-agree-disagree-that-81780.S.52354777
[4] I. Bider, A. Karapantelakis, and N. Khadka, "Building a High-Level Process Model for Soliciting Requirements on Software Tools to Support Software Development: Experience Report," in PoEM 2013, CEUR, Vol. 1023, Riga, Latvia, 2013, pp. 70-82.
[5] Agile Alliance. (2001) Manifesto for Agile Software Development. [Online]. http://agilemanifesto.org/
[6] K. Conboy and B. Fitzgerald, "Toward a conceptual framework of agile methods: a study of agility in different disciplines," in Proceedings of the 2004 ACM workshop on Interdisciplinary software engineering research, Newport Beach, 2004, pp. 37-44.
[7] K. Conboy and B. Fitzgerald, "Toward a conceptual framework of agile methods," in Extreme Programming and Agile Methods-XP-Agile Universe 2004, 2004, pp. 105-116.
[8] Ikujiro Nonaka, "A dynamic theory of organizational knowledge creation," Organization science, vol. 5, no. 1, pp. 14-37, 1994.
[9] I. Bider, P. Johannesson, and E. Perjons, "Design science research as movement between individual and generic situation-problem-solution spaces," in Organizational Systems. An Interdisciplinary Discourse.: Springer, 2013, pp. 35-61.
[10] T. Andersson, I. Bider, and R. Svensson, "Aligning people to business processes experience report," SPIP, vol. 10, no. 4, pp. 403-413, 2005.
[11] R. Conant and R. Ashby, "Every good regulator of a system must be a model of that system," Int. J. Systems Sci., vol. 1, no. 2, pp. 89-97, 1970.
[12] R. L. Flood and and E. R. Carson, Dealing with complexity: an introduction to the theory and application of systems science.: Springer, 1993.
[13] B. Boehm and R. Turner, Balancing Agility and Discipline.: Addison-Wesley, 2004. [14] IbisSoft. (2012a) ibisSoft. [Online]. http://www.ibissoft.se/node/7
[15] Mitch Lacey. (2008a) Agile Aliance. [Online].
http://conferences.agilealliance.org/sessions/20083058
[16] Mitch Lacey. (2010) Mitch Lacey & Associates. [Online]. http://www.mitchlacey.com/resources/working-software-is-not-enough
[17] Mitch Lacey. (2008b) InfoQ. [Online]. http://www.infoq.com/presentations/A-Story-of-Project-Failure-Mitch-Lacey
[18] Michael Krigsman. (2008) ZDNet. [Online]. http://www.zdnet.com/blog/projectfailures/agile-anatomy-of-a-failed-project/1100
[19] Peter Wendorff and Dace Apshvalka, "The Knowledge Management Strategy of Agile Software Development," in ECKM, Reading, UK, 2005, pp. 607-614.
[20] Phillip G. Armour, "The Case for a New Business Model. Is software a product or a medium?," Communications of the ACM August 2000/Vol. 43, No. 8, vol. 43, no. 8, pp. 19-22, 2000.
