Adaptive Software Development

Speculate

In complex environments, planning is a paradox. According to CAS theory, outcomes are unpredictable. Yet wandering around, endlessly experimenting on what a product should look like is not likely to lead to profitability either. “Planning,” whether it is applied to overall product specifications or detail project management tasks, is too deterministic a word. It carries too much historical baggage. “Speculate” is offered as a replacement.

When we speculate, it’s not that we don’t define a mission to the best of our ability. (I use

“mission” as a summary term for objectives, vision, goals, and outline requirements.) It simply admits the obvious — in some important dimensions of our mission statements, we are more than likely wrong. Whether we misread our customer’s needs, or technology changes, or competitors come out with a better mousetrap, the probability of mistakes is high. So let’s be honest, postulate a general idea of where we are going, and put mechanisms in place to adapt. In a complex environment, following a plan produces the product you intended — just not the product you need.

Speculate

Learn Collaborate

Collaborate

Managing in a complex environment is scary as hell — it is also a blast. If we can’t predict (plan), then we can’t control in the traditional management sense. If we can’t control, then a significant set of current management practices is no longer operable, or more specifically, only operable for those parts of the development process that are predictable.

Did u know? What are the uses of collaboration in adaptive software development?

Collaboration, in this context, portrays a balance between managing the doing (the main thrust of traditional management) and creating and maintaining the collaborative environment needed for emergence.

Figure 5.2: The Adaptive Cycle

Software Engineering

Notes As projects become increasingly complex, the balance swings much more toward the latter.

At times it seems almost mystical, but in field after field, from physics to cellular automata to some of my client’s projects, emergence has, well, . . . emerged . We have all experienced emergent results on some special project, but it somehow seemed nearly accidental, not something to count on in a crunch. CAS provides some comfort that it is not accidental.

For a project manager, maintaining this balance means two things. First, he or she has to decide which parts of the project are predictable. For example, we can predict that without appropriate configuration control procedures, a software project of any size can implode. For parts that are unpredictable, he or she has to establish an environment in which the wild and wonderful properties of emergence — basically open, collaborative, messy, exciting, diverse, anxiety-ridden, and emotion-laden — can exist.

Unfortunately, at least for some people, CAS postulates certain conditions for emergent behavior.

The most important is that it happens at the edge of chaos. Whether in physics, biology, business, or human behavior, it appears that there are three broad categories of environments — stable, unstable (or chaotic), and a transition zone labeled the “edge of chaos.”

The edge of chaos is the constantly shifting battle zone between stagnation and anarchy, the one place where a complex system can be spontaneous, adaptive, and alive.

In human terms, being in chaos is analogous to being psychotic. So the trick is to lead a project team away from the familiar and the stable toward chaos, but not all the way. Success comes to those who can hold anxiety, who can attune themselves to paradox and uncertainty. Innovation, creativity, and emergent results are born in the transition zone at the edge of chaos.

Learn

Collaborative activities build products. Learning activities expose those products to a variety of stakeholders to ascertain value. Customer focus groups, technical reviews, beta testing, and postmortems are all practices that expose results to scrutiny.

Learning, according to the dictionary, is gaining mastery through experience. In an adaptive environment, learning challenges all stakeholders, including both developers and customers, to examine their assumptions and use the results of each development cycle to learn the direction of the next. The cycles need to be short, so teams can learn from small rather than large mistakes.

They also need to be double-loop, so teams learn both about product changes and more fundamental changes in underlying assumptions about how the products are being developed.

Speculate—Collaborate—Learn

If you examine the speculate — collaborate — learn cycle, even briefly, it becomes obvious that the three stages overlap. It is difficult to collaborate without learning or to learn without collaborating. They are purposely messy, nonlinear, overlapping terms — how could terms that describe an adaptive framework be otherwise?

For many project leaders and project teams, adaptive development is a terrifying prospect. First, we knock away the foundation pillar of cause and effect, so we can’t say for sure what needs to be done next. Next we force the team into meeting deliverable goals, but we admit we don’t know exactly what they are. Then when they then get anxious and concerned about all the

“seemingly inefficient” groping around for solutions (because of the needed diversity and multiple interactions), we have to say that high anxiety is part of the new game, and it won’t go away. And finally, when a successful product emerges, to many it seems almost accidental. It is not a place for the timid.

Notes Are there organizations practicing adaptive development? I would offer Microsoft as an example

of a company that excels because its management style exemplifies adaptive development.

Notes For all the techniques and practices discussed in books like Microsoft Secrets, we believe Microsoft prospers because at the core of these practices lies the abandonment of determinism and an embrace of the messiness of speculation — collaboration — learning.

Through the Lens of Adaptive Software Development

The lens of ASD offers a different perspective on software management practices. The following is a brief examination of two of these, quality and RAD, both of which have ramifications for gathering requirements.

Do it Wrong the First Time

Using our new lens, let us look at the current state of software quality management practices epitomized by the phrase “Do it right the first time.” In a complex environment, “Do it right the first time” is a recipe for failure.

First, how can we predict what right means? In the early stages, if the delivery time horizon isn’t too far out, we may be able to speculate on what the generally correct direction is, but defining

“right” borders on fantasy. Even if we could define right, doing it the first time makes no sense for other than trivial products. The first time assumes we understand the cause and effect, the specific algorithm of getting to the final product from our initial starting position, and the needs of all stakeholders — it says we know it all.

RAD Practices: A Step in the Right Direction

RAD practices generally involve some combination of the following:

 Evolutionary life cycle

 Customer focus groups, JAD sessions, technical reviews

 Timeboxed project management

 Continuous software engineering

 Dedicated teams with war rooms

Most RAD projects I’ve been associated with over the last five years have had an adaptive, emergent flavor. More recently, as my practice has focused on speculating and establishing collaborative environments, emergent characteristics have become even more pronounced.

RAD is an anathema to many IT organizations. It works, but many would prefer it did not. Their defense is to label RAD as “hacking” and/or to relegate it to small (and by implication relatively unimportant) development projects. But Microsoft and others have produced incredibly large and complex software using techniques comparable to RAD. RAD is scary to traditional organizations because it raises questions about their fundamental world view.

Example: RAD practices and the Microsoft process are both examples of adaptive development in action. Giving them a label (i.e., adaptive development) and realizing there is a growing body of scientific knowledge (i.e., CAS theory) that begins to help explain why they

Software Engineering

Notes

5.2.3 DSDM

Dynamic Systems Development Method (DSDM) is a structure based initially around Rapid Application Development (RAD), carried by its continuous user involvement in an iterative development and incremental approach which is approachable to changing requirements, in order to expand a system that meets the business needs on time and on budget. It is one of a number of Agile methods for developing software and forms part of the Agile Alliance.

DSDM was developed in the United Kingdom in the 1990s by a group of vendors and experts in the field of Information System (IS) development, the DSDM Consortium, unite their best-practice experiences. The DSDM group is a non-profit and vendor independent association which owns and administers the framework. The first version was finished in January 1995 and published in February 1995. The current version in use at this point in time (April 2006) is Version 4.2: Framework for Business Centered Development released in May 2003.

As an expansion of rapid application development, DSDM focuses on Information Systems projects that are distinguished by fixed schedules and budgets. DSDM addresses the ordinary reasons for information systems project failure counting exceeding budgets, missing deadlines, and lack of user participation and top management commitment.

DSDM identifies that projects are limited by time and resources, and plans consequently to meet the business needs. In order to attain these goals, DSDM encourages the use of RAD with the ensuing danger that too many corners are cut. DSDM applies some principles, roles, and techniques.

Notes In some circumstances, there are possibilities to integrate practices from other methodologies, such as the Select Perspective, Extreme Programming (XP), and PRINCE2, as complements to DSDM. Another agile method that has some similarity in process and concept to DSDM is Scrum.

Principles of DSDM

There are 9 fundamental principles of DSDM consisting of four foundations and five starting-points for the structure of the method. These values form the keystone of development using DSDM.

 User participation is the main key in running a well-organized and effectual project, where both users and developers share a workplace, so that the decisions can be made precisely.

 The project team must be authorized to make decisions that are important to the progress of the project, without waiting for higher-level approval.

 DSDM focuses on recurrent delivery of products, with supposition that to deliver something

“good enough” earlier is always better than to deliver everything “perfectly” in the end.

By delivering product frequently from an early stage of the project, the product can be tested and reviewed where the test record and review document can be taken into account at the next iteration or phase.

 The main criteria for acceptance of deliverable in DSDM are on delivering a system that addresses the current business needs. It is not so much directed at delivering a perfect system addressing all possible business needs, but focuses its efforts on critical functionality.

Notes

 Growth is iterative and incremental, driven by users’ feedback to converge on an effective business solution.

 All changes during the growth are reversible.

 The high level scope and requirements should be base-lined before the project starts.

 Testing is carried out all through the project life-cycle.

 Communication and co-operation among all project stakeholders is required to be efficient and effective.

Prerequisites for using DSDM

In order for DSDM to be an accomplishment, a number of fundamentals need to be realized.

First, there needs to be interactivity between the project team, future end users and higher management. This addresses well known breakdown of IS development projects due to lack of top management motivation and/or user involvement.

The second significant prerequisite for DSDM projects is the decomposability of the project. The possibility of decomposition into smaller parts allows the iterative approach, and activities, that are hard to prioritize, often causes delays. And that is accurately the effect that DSDM was developed to avoid. Another group of projects for which DSDM is not well-suited are safety-critical ones. The extensive testing and validation found in these kinds of projects collide with DSDM goals of being on time and on budget. Finally, projects that aim at reusable mechanism might not be well-suited for development using DSDM, because the demands on perfection are too high and collide with the 80%/20% principle described earlier.

The Phases of DSDM

The DSDM structure consists of three sequential phases, namely the pre-project, project life-cycle and post-project phases. The project phase of DSDM is the most detailed of the three phases.

The project life-cycle phase consists of 5 stages that form an iterative step-by-step approach in developing an IS. The three phases and corresponding stages are explained extensively in the subsequent sections. For each stage/phase, the most significant activities are addressed and the deliverables are mentioned.