Bridging the software quality gap

(1)

Bridging the software quality gap

Markus Lindgren

Department of Computing Science, Ume˚

a University

October 2012

(2)

Abstract

There is a gap in the understanding of software quality between develop-ers and non-technical stakeholddevelop-ers, the software quality gap, which leads to disagreements about the amount of time that should be used for quality im-provements. The technical debt metaphor reduces this gap some by describing software quality in economic terms, enabling developers and non-technical stake-holders to communicate about the quality. However, the metaphor is vague and not very concrete in explaining the gap. The purpose of this thesis is to con-cretize the technical debt metaphor using Domain-Driven Design, an approach in which communicating software characteristics is central, in order to reduce the software quality gap further.

Using the terminology of Domain-Driven Design, a new concept is defined:

model debt, which measures and communicates the software quality of the do-main model. An application is built, theModelDebtOMeter, which extracts the domain model of a software system and visualizes it along with its corresponding model debt. The model debt of a legacy system is amortized and the results of the amortizations are presented to the non-technical stakeholders of the system using theModelDebtOMeter, allowing the non-technical stakeholders to evaluate the model debt concept.

The results show that the non-technical stakeholders think that model debt is an understandable concept which reduces, but not eliminates, the software quality gap.

(3)

Introduction

The product owner entered the room. The team was already sitting around the table. It was time for another sprint planning meeting. The scrum master spoke:

- What activities did you have in mind for this sprint?

- Well, let’s see, first we have to add system support for modifying the layout of our invoices. The marketing department wants to have the possibility to customize the advertisements depending on the size of the company, its current engagement with us and so on. We have a couple of minor bugs that we need to get fixed. Then it would be great if we could start working on the next release. - I see. We have a few activities that we didn’t have time to complete in the previous sprint that we really need to get done. We have to restructure the business rules engine a little. We realized in the last sprint that it’s just taking way too long to add new rules. We think it will be about five story points.

- But we won’t be adding new rules anytime soon, right?

- That’s true, but I still think we need to restructure it. Besides, we worked on the rules engine last sprint, so we have a really clear picture of what to do.

The product owner thought for a while about what the scrum master had said. He was really tired of these kinds of discussions. Restructuring this and restructuring that. Why don’t we spend all of our time restructuring? Too bad developers can’t stop worrying about technical issues and instead focus on business value. It doesn’t matter anyway. The management made themselves very clear this morning. They explicitly said that we had to start working on the next release.

- Well, I’m still not convinced. I really think we need to get started on the next release.

The scrum master was really tired of these kinds of discussions. The next release, and then the next release, and then the next release. Why don’t we stop designing, testing and even thinking so that we can release quicker? The road to hell, that’s what would happen. Too bad business people are so obsessed with making new releases that they neglect software quality. The scrum master was about to give up the argument but thought that he better try the technical debt metaphor first.

- You know that ignoring to restructure the business rules engine will increase our technical debt. Don’t you think we’re paying enough interest as it is?

(6)

added something to the usually so meaningless discussions about restructuring activities that he had from time to time with the developers, but he still didn’t feel confident with this new kind of invisible debt.

- I guess you’re right. But unfortunately there isn’t enough time to do it now. It will have to wait to a future sprint.

The scrum master felt the frustration coming. He had experienced this too many times: trying to explain the need for quality improvement to the product owner. The technical debt metaphor had made this task easier but now it almost seemed to make things worse. It didn’t explain software quality good enough. It wasn’t concrete enough. It didn’t bridge the software quality gap between developers and non-technical stakeholders.

1.1 Approach

There is a gap in the understanding of software quality between developers and non-technical stakeholders, the software quality gap. The technical debt metaphor reduces this gap some by introducing a shared language that enables developers and non-technical stakeholders to communicate about software qual-ity. However the metaphor is a very rough and uncut technique and it is not very concrete in explaining the gap. What is needed is a better developed shared language that communicates software quality in a more precise manner.

One way to tackle this problem is to look closer at other approaches within software development in which communicating software characteristics is central to see if they can be used to concretize the technical debt metaphor. This thesis takes this approach and does so by looking closer at Domain-Driven Design: an approach in which it is central that developers and non-technical stakeholders design a software system in close collaboration.

1.2 Related work

I have not found any attempt to explore the technical debt metaphor using Domain-Driven Design. There are however ongoing attempts to explore the metaphor: “Therefore, we propose managing technical debt as a part of the future research agenda for the software engineering field.” [3]. The software quality gap is also recognized and described: “Software developers and corporate managers frequently disagree about important decisions regarding how to invest scarce resources in development projects, especially in relation to internal quality aspects that are crucial to system sustainability, but that are largely invisible to management and customers... Engineers often advocate for such investments, but executives question their value and frequently decline to approve them, to the long-term detriment of software projects.”. The approach to abandon the technical debt metaphor in favor of another technique is also recognized: “Is debt a sound metaphor for managing expedients and remediative investments in software projects? If not, is there a closely related metaphor that is better‘”.

(7)

1.3 Avanza Bank

Avanza Bank is one of the biggest actors for trading shares and funds in Swe-den. All transactions are handled electronically and the business contains many crucial software systems. Software quality is very important, not only for de-velopers but also for non-technical stakeholders, since IT is such a crucial part of the business. Avanza Bank has made previous efforts to communicate soft-ware quality between developers and non-technical stakeholders using aquality barometer.

1.3.1 The quality barometer

The quality barometer is an estimation of software quality made by developers in a dialogue with non-technical stakeholders. The estimation is presented using a business-friendly scale which signals how the customers of Avanza Bank are af-fected by the software quality. The quality barometer has been used successfully for a couple of years at Avanza Bank.

1.3.2 This thesis and Avanza Bank

The problem with the quality barometer is that it is built upon estimations. It would be better if software quality could be based on more concrete facts but still be explainable to non-technical stakeholders. This thesis takes one approach to see if this is possible.

1.4 Outline

Background information of technical debt and Domain-Driven Design are pre-sented in chapter 2 and 3. A more specified purpose is defined in chapter 4. The concepts given in the background information are used to define a new concept:

model debt in chapter 5. Chapter 6 outlines the method used in this thesis and chapter 7 summarizes the results obtained after applying the method. In chapter 8 the result are discussed and conclusions are drawn.

(8)

Chapter 2

Technical debt

In the last couple of years technical debt has become a new buzzword in the agile community. As [3] puts is “The technical debt metaphor is gaining significant traction in the agile development community...”.

Despite its boom in use lately, the technical debt metaphor is almost two decades old. It was coined in 1992 by Ward Cunningham in an experience report [4]:

“Shipping first time code is like going into debt. A little debt speeds devel-opment so long as it is paid back promptly with a rewrite. [...] The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation, object-oriented or otherwise.”

2.1 The debt confusion

The original description of technical debt was not very precise and when the metaphor gained in popularity, people started to interpret it in different ways. This made Cunningham further clarify what he meant [5]:

“I thought borrowing money was a good idea, I thought that rushing software out the door to get some experience with it was a good idea, but that of course, you would eventually go back and as you learned things about that software you would repay that loan by refactoring the program to reflect your experience as you acquired it. I think that there were plenty of cases where people would rush software out the door and learn things but never put that learning back into the program, and that by analogy was borrowing money thinking that you never had to pay it back. [...] You know, if you want to be able to go into debt that way by developing software that you don’t completely understand, you are wise to make that software reflect your understanding as best as you can, so that when it does come time to refactor, it’s clear what you were thinking when you wrote it, making it easier to refactor it into what your current thinking is now.”

The bottom line of what Cunningham is saying is that technical debt does not incur when you are developing software but after the software has been released, when you learn things about it by getting feedback from reality.

(9)

“definition” of technical debt to also include debt that comes from poorly written code during development. The danger arises when you combine the expanded definition with Cunningham’s view that technical debt is something desirable to have after a release. This is something that Ward also recognizes:

“A lot of bloggers at least have explained the debt metaphor and confused it, I think, with the idea that you could write code poorly with the intention of doing a good job later and thinking that that was the primary source of debt. I’m never in favor of writing code poorly, but I am in favor of writing code to reflect your current understanding of a problem even if that understanding is partial.”

2.2 Other views on technical debt

Steve McConnell categorizes technical debt into unintentional debt, which is foolish to incur, and intentional debt, which might be incurred for reasons such as time to market, preservation of startup capital and delaying development expense [10]. Part of the intentional debt would be equal to Cunningham’s original definition.

Martin Fowler further divided intentional and unintentional debt into reck-less and prudent debt [8].

Robert C. Martin comes close to Wards original definition by saying “A mess is not a technical debt” [9] but he also diverges from Ward by including technical frameworks into the discussion.

2.3 Communication is the key

A clear definition of technical debt seem to be lacking. This is most likely a result of the fact that most writing on the topic has been in blogs and that no research has been carried out in order to develop such a definition.

Given the vague definition of technical debt, it is fascinating how widespread the metaphor has become. I think that the reason for this is that the metaphor addresses a very real and significant problem in the software industry which is screaming for solutions: the software quality gap.

The power of the technical debt metaphor is not that it provides a good way to measure software quality. The power of the technical debt metaphor is that it reduces the software quality gap some by describing software quality to non-technical stakeholders in economic terms. This is something that [3] also recognizes: “Technical debt metaphor to date mostly has been used as a communication device.”. The problem with the metaphor is that is doesn’t reduce the software quality gap enough.

(10)

Chapter 3

Domain-Driven Design

The term Domain-Driven Design (DDD) was coined by Eric Evans in his book with the same title [7]. This chapter is a summary of the parts of the book relevant to this thesis. As the subtitle of the book suggests DDD is about

Tackling Complexity in the Heart of Software. What is meant by this heart of software?

3.1 Focus on the domain instead of technology

A common view is that software development is about solving different technical challenges. The complexity lies in choosing a suitable technology for the task at hand and in creating a design which makes the different technologies fit together smoothly. This focus is natural since many developers have a big interest in technology.

DDD recognizes that the main complexity in software development is not technical. Instead the complexity lies in the area for which the software is being developed: the domain of the software. A software domain can be anything from cargo shipping to accounting. This domain is what DDD calls the heart of the software. A domain expert is a person who has good knowledge of the domain, e.g. in the domain of accounting an accountant is a domain expert.

When focus is changed from technology to the domain new problems arise. How should a complex domain be managed in a software system? Fortunately DDD has a solution to this problem: knowledge crunching the minds of the domain experts into a domain model.

3.2 Knowledge crunching

Why can’t we ask the domain experts for the domain model? Why is this knowledge crunching thing necessary? The reason for this is that the knowledge of the domain experts contains lots of common sense and developing software from common sense is impossible. The common sense have to be concretized in order to make software development possible. This process is what Evans calls knowledge crunching. Developers together with domain experts distill the complex and unstructured knowledge of the domain experts into rigorous concepts which form a domain model. This is an iterative process which never

(11)

completes. New ideas are constantly tested and many of them are rejected. Knowledge crunching increases the domain knowledge of the developers and gives domain experts better insight into how rigorous computer software have to be.

When focus of software development lies on technology rather than on the domain little knowledge crunching is done together with the domain experts. The communication between developers and non-technical stakeholders is about what the system should do and not about how the domain works. The domain experts describe what they want and the developers implements it. As Evans puts it: “...if programmers are not interested in the domain they learn only what the application should do, not the principles behind it.”.

Typical things that will happen when knowledge crunching is carried out is that new concepts will appear. Concepts that the domain experts use in their daily jargon but that the developers haven’t understood or thought was important. To make this happen it is important that the developers carefully listen to the language of the domain experts. Are they using terms that seem complex but still important? Do they look puzzled when you use your own terms? Do they correct you? An example of this phenomenon is that developers often use the word user to describe any person using the system. When talking to domain experts one often discovers that this user is actually something more specific, e.g. a customer or a person with a power of attorney.

3.3 Domain Model

Evans makes a good explanation of a model: “A model is a simplification. It is an interpretation of reality that abstracts the aspects relevant to solving the problem at hand and ignores extraneous detail.”. Important points are that a model is a simplification: it is not meant to capture all aspects of reality and that it is an interpretation of reality which means that it can be interpreted in a infinite number of ways. The goal of the knowledge crunching of the domain experts minds is to end up with a domain model which contains abstractions to solve the problem at hand in the most efficient way. To achieve this goal the concepts of the domain model have to be very clear and exact. Stated in another way, the concepts of the domain model need to have a high conceptual clarity.

A typical sign of an ineffective domain model is when developers start to talk about corner cases. When there are many corner cases in a software system it is very likely that the domain model, if present at all, contains poor abstractions for solving the problem at hand. In this case developers often blame the domain experts for coming with tedious requirements, but the real problem is that the developers don’t try to adapt the domain model to match the requirements of the domain experts. Another sign of this problem is that the design feels awkward in such a way that a seemingly easy problem takes a lot of work to solve. A warning sign is when domain experts use a vocabulary that is not present in the implementation.

Creating a domain model is a highly creative process. To do it well it is necessary to be able to think outside the frames of the existing model and not to be locked up in the present concepts. It is also important not to try to force the evolution of the domain model into a certain direction. According to Evans,

(12)

domain models are often not as obvious as one might think when first faced with a problem.

3.4 Model and implementation

For a domain model to be useful it has to be used in the running software system. After all, the purpose of the domain model is to make the problem at hand easier to tackle by using the knowledge of the domain experts. If the domain model isn’t present in the software that advantage is lost and the whole purpose of DDD vanishes.

The domain model and implementation shape each other. If a knowledge crunching session leads to a discovery of a new concept the implementation have to be refactored to include that concept. In the same way, if a developer finds a limitation in a technical framework which make the implementation of the current domain model impossible, that limitation has to be communicated to the domain experts and the domain model has to be modified to reflect it.

To be able to make the gap between the domain model and the implementa-tion as small as possible it is necessary that the implementaimplementa-tion language allows you to create concepts similar to those in the domain model. The object-oriented languages are such languages since they are based on a modeling paradigm: the object-oriented modeling paradigm. Furthermore, the object-oriented paradigm often feels natural to both developers and non-technical stakeholders. This makes object-oriented programming and DDD a good fit.

3.5 Ubiquitous language

It is not enough for the domain model to be present in the implementation. It has to be present in all activities which has anything to do with the domain such as diagrams, in writing and in speech. The language of the domain model should be the basis for all communication about the domain. It should be ubiquitous. No big surprise Evans calls it theubiquitous language.

The reason for it to be ubiquitous is that translation makes communication very cumbersome. Developers communicate using a technical language while non-technical stakeholders communicate using their business language. The gap between the two languages is big and a significant translation effort has to be made to be able to communicate at all. Even so, confusion and misunderstand-ings regularly occur. In the best case someone who knows both the jargon of the domain experts and the technical language of the developers are able to make inexact translations. In the worst case a developer or a domain expert stops listening to the other part because they simply don’t understand each other. As Evans puts it: “...developers have to translate for domain experts. Domain experts translate between developers and still other domain experts. Developers event translate for each other. [...] This leads to unreliable software that doesn’t fit together.”.

Of course, domain experts and developers can use their own jargon at times, but not when talking about the domain.

(13)

3.6 Isolate the domain

To be able to tackle a new business requirement effectively it is necessary to have a clear picture of the part of the software that the requirement will affect so that it is possible to reason about the relationship between that part and the requirement. A business problem usually has little to do with technology. If the business logic of the software is tangled with technical infrastructure then it is very hard to get that clear picture. Therefore it is necessary to dedicate an explicit part of the software in which the business logic lives: the domain layer. The layering pattern will not be explained here. The important point is that the business logic lives in the domain layer and that the domain layer have no dependencies to other layers. The domain layer is a prerequisite to be able to apply DDD.

3.7 Bounded context

At most companies there is not one domain model at play, but multiple. The failure to recognize this fact and trying to apply one domain model across the entire company will come at a high price. Concepts from different models will be mixed up causing confusion and bugs. Concepts will appear multiple times making changes more difficult. The unified model will start to fragment. A typical scenario is that a developer changes a concept without establishing the new meaning to everyone else involved and thus failing to include it in the ubiquitous language. This happens simply because it’s too time consuming to integrate a change with everyone involved on a project.

One have to be aware of these models and define their relationships to each other. To do that Evans introduced the concept of a bounded context: “The bounded context is whatever set of conditions must apply in order to be able to say that the terms in a model have a specific meaning.”. A model always applies in a context. The bounded contexts should be explicitly defined, both in the domain model and in the implementation. It is natural that the bounded contexts follow the organization of teams since people who work close together often share the same concepts.

When multiple bounded contexts are in play new questions arise. How should the different contexts relate to each other? Which contexts needs to integrate with each other and which can diverge? Is it possible to translate a concept from one context to another? Evans presents a number of strategic design patterns for dealing with these kinds of problems but this is out of the scope for this thesis.

3.8 Scenarios

A scenario is a concrete example of something that happens in the domain and that is important to the domain experts, together with its preconditions and postconditions. Let’s look at an example:

Given a company.

And a manager who works at the company.

(14)

When the manager fires the employee.

Then the employee should have a salary of 0$.

In this example the preconditions follows theGivenkeyword, what happens in the domain follows the When keyword and the postconditions follows the

Thenkeyword. Writing scenarios using these keywords is not required, but it is a common practice which comes from agile acceptance testing, a methodology described in the next section.

Scenarios are created in collaboration between developers and domain ex-perts and are stated in the ubiquitous language of the domain model. In the example above company, manager, employee, salary and fires are all part of the ubiquitous language of that domain model.

The purpose of scenarios is to encourage knowledge crunching. Walking through a concrete example forces you to think about the concepts of the domain model in a very concrete way. Evans doesn’t describe scenarios at any greater depth in [7] but talks about their role in DDD and in the agile development process in his whirlpool model [6].

Scenarios are not only important within DDD. They play the main role in agile acceptance testing.

3.9 Agile acceptance testing

Agile acceptance testing (AAT) is a methodology related to DDD. It recognizes the need for an ubiquitous language but has a different focus than DDD. It focuses almost entirely on creating scenarios through collaboration and commu-nication between developers and non technical-stakeholders. Agile acceptance testing is described by Gojko Adzic in [2].

Adzic describes a communication gap, which is similar to the software qual-ity gap but less specific since it includes all communication problems between developers and non-technical stakeholders, not just the ones related to software quality. The purpose of AAT is to reduce this gap by improving the communi-cation between developers and non technical stakeholders. This is achieved by writing scenarios.

Apart from focusing more clearly on scenarios, AAT distinguishes itself from DDD by recognizing the need to automate scenarios by implementing them as acceptance tests. Together the implemented scenarios form an executable specification which contrary to an ordinary requirements specification always stays up to date since it constantly executes and verifies that the system behaves as specified.

Although AAT recognizes the need for an ubiquitous language it does not require it to be present in the implementation. Adzic writes: “Developers then implement the domain code and the test automation part in parallel, with glue code to connect them. This glue code allows us to effectively separate the speci-fication from implementation details.” [2, p. 111]. A relevant question is if the language really is ubiquitous if it is not present in the implementation. Anyhow, this is something that clearly separates AAT from DDD.

(15)

Chapter 4

Purpose

It is clear from the background chapter of DDD that a weak domain model with a weak ubiquitous language will lead to a communication gap between developers and non-technical stakeholders. This gap is closely related to the software quality gap described in the introduction. How can it be possible for a non-technical stakeholder to know about the quality of a software system when they fail to communicate its design with the developers? It is reasonable to assume that by concretizing and explaining this communication gap to non-technical stakeholders the software quality gap will also decrease. This is the approach taken by this thesis and it is done by joining the concepts of DDD with the technical debt metaphor.

4.1 Purpose

The purpose of this thesis is to use Domain-Driven Design to concretize the technical debt metaphor so that software quality for a non-technical stakeholder becomes visible, measurable and understandable.

4.2 Research questions

To fulfill the purpose of this thesis the research questions below have to be answered. It is assumed that the concretization of the technical debt metaphor will lead to a new software quality concept.

• Which concepts from DDD should be used to concretize the technical debt metaphor? How will they measure software quality?

• How should the domain model and the new software quality concept be presented in order to make software quality visible? How can the domain model be extracted from an existing software system?

• Does the new software quality concept work in practice? Can it be applied to a working software system? Does it measure software quality in a reasonable way?

(16)

• In what way can the new software quality concept be tested to make sure that software quality becomes understandable for a non-technical stakeholder?

(17)

Chapter 5

Model debt

The first research question will be answered in this chapter by introducing the new concept ofmodel debt. Model debt concretizes the technical debt metaphor by using concepts from DDD and adding measurability.

The strong focus on the domain which DDD requires has implications on software quality. A strong focus on the domain makes the quality of the do-main model much more important. The role of the non-technical stakeholders in software development also changes. They become more important since they are the persons with the best knowledge in the domain for which software is going to be developed. The consequence of this thinking is that software qual-ity is something that not only concerns developers but also the non-technical stakeholders. The quality of the domain model is more closely related to the non-technical stakeholders and the business than to developers and software. This is the idea behind the concept of model debt and it explains why the names and the explanations of the model debt parts are free of software terms. Model debt applies to the domain model of a software system and consists of three types:

• Business isolation debt. Indicates how well the business logic is isolated from technical infrastructure and irrelevant business logic.

• Business behaviour debt. Indicates the amount of knowledge crunching carried out in the domain model.

• Inefficient business debt. Indicates how efficiently the current domain model solves problems in the domain.

These concepts will be described in further detail in the rest of this chapter. The descriptions will introduce some new concepts:

Model block - An object in the domain model.

Criteria - The criteria which a model has to fulfill in order to be free of debt.

Measurement - Description of how the debt is measured.

(18)

5.1 Business isolation debt

Keeping the business logic isolated is a necessity. If the business logic is tan-gled with technical infrastructure or if different parts of the business are mixed together it is very difficult, if not impossible to understand how the business works. A low business isolation debt is a prerequisite to be able to amortize business behaviour debt.

Criteria

All model blocks belong to one layer and one bounded context. Model blocks in the domain layer don’t have associations to model blocks in other layers.

Measurement

Business isolation debt is measured by the number of model blocks which don’t meet the criteria above. If a model block belongs to a bounded context but not to a layer or vice versa, half of the model block meets the criteria. If a model block has an association to another model block with business isolation debt, half of the model block meets the criteria.

Interest

A low business isolation debt is a prerequisite for amortizing business behaviour debt and inefficient business debt. Without an isolated business it is not possible to walk through scenarios with the domain experts since it is not known where the domain is and it is hard or impossible to reason about the efficiency of the domain model. Choosing not to amortize business isolation debt means that you’re stuck with a technical model which is not possible to communicate to non-technical stakeholders.

Business isolation debt means that the bounded contexts of the domain model are not explicitly defined. This leads to, as described in the background, a fragmented and confusing domain model

5.2 Inefficient business debt

To concretize the technical debt metaphor further we use the approach described in [3]: “One way to understand technical debt is as a way to characterize the gap between the current state of a software system and some hypothesized “ideal” state...”. Using concepts of DDD these states will be two models:

The current model - The current model in use and the model for which you want to know the model debt.

The desired model - The domain model which the non-technical stakeholders want, but which doesn’t yet exist.

An interesting property of inefficient business debt is that there is no debt without a desired model. This means that if all stakeholders are willing to speak the ubiquitous language of the existing model then there is no model debt in

(19)

the system. In reality this will seldom be the case since this means that non-technical stakeholders have to learn the vague non-technical language that is most likely present in the system.

When the domain experts are aware of flaws in the current model which don’t get corrected the model contains inefficient business debt. Note the similarities between inefficient business debt and Cunningham’s definition of technical debt.

Criteria

The current model equals the desired model, i.e. the models don’t differ in any of the following ways:

• A model block in the current model belongs to another layer in the desired model.

• A model block in the current model belongs to another bounded context in the desired model.

• A model block in the current model doesn’t exist in the desired model. In other words it is redundant.

• A model block which exists in the desired model is missing in the current model.

Measurement

Inefficient business debt is measured by the number of model blocks which differs between the current model and the desired model in any of the ways described in the criteria above.

Interest

The greater the gap between the current model and the desired model, the greater the communication gap between developers and non-technical stake-holders. This gap leads to misunderstandings and many of them will in turn lead to bugs. A big communication gap also indicates that little knowledge crunching has been applied to the current model. This means that the con-cepts in the model most likely are technical in nature and not the most efficient concepts to solve problems in the domain. Software development becomes in-efficient. It is difficult to know what the non-technical stakeholders want and even if that is understood it takes a long time to implement it since the current model is inefficient.

5.3 Business behaviour debt

To make inefficient business debt useful the desired model has to be known. How can that be achieved in practice? Looking back at DDD we find an answer: knowledge crunch scenarios together with domain experts using the ubiquitous language of the current domain model. Doing that will reveal weaknesses in the current model which the domain experts will want to replace with modified concepts in a new desired model. Another way to put it is that if you don’t

(20)

have an executable specification covering the entire domain model then you don’t know how far away the current model is from the desired model. The current model contains inefficient business debt which you can’t see. Therefore, a low business behaviour debt is a prerequisite to be able to amortize inefficient business debt.

Criteria

All behaviour of the domain model is exercised by an executable specification.

Measurement

Business behaviour debt is measured by the number of behaviours which are exercised by an executable specification.

Interest

If the business behaviour debt is high it is very likely that the software does not work as expected, i.e. it contains bugs. This is because the developers will have implemented the business behaviour without communicating it to the non-technical stakeholders, causing misunderstandings and in turn bugs. An-other advantage of keeping the business behaviour debt low is that you gain advantages from the executable specification, e.g. when you make a change to the software system you can be certain that the domain model still work as expected.

The process of amortizing business behaviour debt reveals the desired model. Therefore, a high business behaviour debt makes it impossible to amortize inef-ficient business debt. This means that you are stuck with an inefinef-ficient domain model.

5.4 Relation between the model debt types

To get anywhere you have to start to isolate the domain model from technical infrastructure and unrelated business logic, i.e. you have to amortize business isolation debt. Doing so will reveal where the domain model is and it is possible to discuss it with non-technical stakeholders. It is possible to walk through scenarios with them and to implement them as an executable specification, i.e. amortizing the business behaviour debt. Doing so will reveal weaknesses in the domain model such as vague or technical concepts, or in other words it will reveal the inefficient business debt. The last step is to amortize this debt so that the efficiency of the domain model increases, making software development faster and less error prone.

(21)

Chapter 6

Method

To answer the remaining research questions I will:

• Build an application, theModelDebtOMeter, which shows the model debt of a software system.

• Amortize the model debt of an existing legacy system at Avanza Bank.

• Let the non-technical stakeholders of the legacy system evaluate model debt to find out if they think that it is an understandable concept.

The last two steps will be explained in more detail below.

6.1 Amortize model debt

First, a legacy system at Avanza Bank suitable for quality improvement has to be identified. This system will from now on be referred to as the legacy system. The first type of model debt that will be amortized will most likely be busi-ness isolation debt since this is a prerequisite for amortizing inefficient busibusi-ness debt and business behaviour debt.

With the business isolation debt reduced it is technically possible to write an executable specification and it is also possible to do knowledge crunching of the domain model together with the domain experts. This knowledge crunching of the current model will be done one step at at time, slowly revealing the desired model:

1. Write a scenario using the ubiquitous language of the existing model to-gether with the domain experts.

2. Find out which problems the domain experts see with the existing model, i.e. reveal the inefficient business debt.

3. Implement the scenario as part of an executable specification and make the scenario pass, i.e. amortize the business behaviour debt.

(22)

6.2 Evaluation

In order to find out if the non-technical stakeholders of the legacy system thinks that model debt is an understandable concept they will be invited to a session where I will:

• Explain the model debt concept using the descriptions given in chapter 5.

• Show how the model debt has changed over time in the legacy system using theModelDebtOMeter.

• Evaluate the model debt concept by letting the non-technical stakeholders fill in the evaluation form found in appendix A.

(23)

Chapter 7

Results

The results of this thesis consist of three parts:

• TheModelDebtOMeter, an application which shows the model debt of a software system.

• The result of the amortizations of model debt in the legacy system at Avanza Bank.

• The result of the evaluation of the understandability of the model debt concept.

Each of these results are described in more detail below.

7.1 The

ModelDebtOMeter

The ModelDebtOMeteris a Java application which shows the model debt of a software system. The application consists of two parts: the importer and the analyzer.

7.1.1 The importer

The importer has the responsibility to import the domain model of an existing software system. To achieve this it needs to be fed with the following parameters:

• The naming convention used for the domain layer.

• The naming convention used for the bounded contexts.

• The location of the executable specification (if there is any).

• The import date, i.e. the date at which the domain model to be imported was in use.

The importer uses Java reflection to obtain information about the classes of the domain layer of the software system. More specifically it obtains:

(24)

• Public methods (behaviour).

• References (through constructors and methods).

• The result of running the behaviour specification.

To be able to obtain this information and to run the behaviour specification, the classes of the domain model have to be available on the class path. This is achieved by a script which:

1. Fetches the source code of the software system at the given import date. 2. UsesMavento compile the source code and package it into jar-files. 3. Invokes the importer with the jar-files on the class path.

Since the importer uses an import date, it is possible to import the same domain model multiple times, each with a different import date. This makes it possible for the analyzer to compare and analyze different versions of the domain model.

The information obtained from the class path is arranged into Java-objects and stored in aneo4jgraph database.

7.1.2 The analyzer

The analyzer has the responsibility to visualize the imported domain models together with their model debt. Upon startup the analyzer loads all the im-ported domain models from the neo4jgraph database and displays them in a menu. When a user selects one of these domain models from the menu it will be visualized in a screen similar to the one shown in appendix B.

The grey box named isolated business contains all model blocks which belong to the domain layer and which are free of business isolation debt. These are the model blocks of interest for business behaviour debt and inefficient business debt. Outside the isolated business are model blocks for which it is not known which layer or bounded context (or both) they belong to, i.e. they contain business isolation debt.

To the right of the domain model the model debt of the domain model is shown by displaying the measurements of the three model debt types. Clicking on one debt type will show more detailed information about that debt type:

Business isolation debt - Displays the model blocks which contain business isolation debt in red. Associations from the domain layer to other layers are also displayed in red since they cause business isolation debt.

Business behaviour debt - Shows all behaviours of the model blocks in the domain layer and displays the behaviours with business behaviour debt in red.

Inefficient business debt - Displays the model blocks which are present in the current domain model but is missing in the most recent domain model in red. Model blocks which are present in the most recent domain model but are missing in the current model are also listed in red. All of these model blocks contain inefficient business debt.

(25)

7.2 Amortization of model debt

In discussion with my supervisor we found a legacy system suitable for quality improvement. It is a software system which is relatively independent of other systems, making it easy to modify without affecting too many other systems. This section describes the results of amortizing the model debt of the legacy system as measured by theModelDebtOMeter. During the amortizations of the model debt, interest that was payed on the debt before the amortizations were made was discovered. This interest is also presented.

7.2.1 Business isolation debt

The first obvious problem of the legacy system was that its business logic was tangled with technical infrastructure and unrelated business logic. In other words, the business isolation debt was high.

Measuring the model debt with theModelDebtOMeterbefore the amortiza-tion yielded the following result:

Business isolation debt 99 of 99 (99%) Business behaviour debt 0 of 0 (0%)

Inefficient business debt 0

The fact that 99 of 99 doesn’t give 100% is not a bug. For one model block, half of its business isolation debt had already been amortized. The reason why the business behaviour debt measures to 0 of 0 is because the model blocks of the domain layer are not known and therefore the behaviours of the model blocks of the domain layer are not known.

The business isolation debt was amortized by moving the legacy system into its own bounded context, separating it from unrelated business logic, and by identifying the model blocks belonging to the domain layer and moving them into that layer. Measuring the model debt with theModelDebtOMeterafter the amortization yielded the following result:

The business isolation debt has been reduced significantly. Note that the total number of model blocks have also been reduced by 37 (from 99 to 62). The reason for this is that unrelated business logic that used to be tangled with the business logic of the legacy system has been moved into its own bounded context. The business isolation debt has not been completely amortized because some dependencies from the domain layer were located outside of the legacy system and it was simply too much work to refactor them into a domain part and a technical infrastructure part.

Interest

Before the amortization, the business logic of the legacy system was tangled with unrelated business logic. This led to the fact that when the business logic

(26)

of the legacy system failed, the business logic of the unrelated business also failed, making it impossible for the users of the systems built upon that logic to keep working as normal. When the unrelated business logic was separated from the legacy system into its own bounded context, the problem disappeared. In other words, when the business isolation debt was amortized, the interest payed on the debt decreased.

7.2.2 Business behaviour debt

With the business isolation debt reduced, business behaviour debt was the next candidate for amortization. In discussion with the non-technical stakeholders we decided to amortize the business behaviour debt of 2 of the total 24 behaviours of the domain model. These two behaviours were chosen because they belong to model blocks which are relatively independent of the rest of the domain model of the legacy system, making it easier to write an executable specification, and also because the non-technical stakeholders had experienced problems with them.

For each of the two behaviours I prepared a few basic scenarios and invited the non-technical stakeholders of the legacy system to a session with the purpose to knowledge crunch the scenarios, come up with new ones, and find out what main problems the non-technical stakeholders experienced in the current domain model.

The result of the session was 7 scenarios which all of them were imple-mented as an executable specification. Measuring the model debt with the

ModelDebtOMeterafter the amortization yielded the following result:

The business behaviour debt decreased from 100% to 92% since two be-haviours were covered by an executable specification. The number of model blocks of the business isolation debt increased from 62 to 63 because new func-tionality had been developed in the legacy system between the two measure-ments.

Interest

Of the 7 scenarios, 2 failed when implemented as an executable specification. The legacy system simply did not work the way the non-technical stakeholders expected. It was not serious defects but still defects that could potentially make the legacy system produce incorrect results. This is a clear example of bugs which are not known because too little knowledge crunching of the domain model has been carried out.

The executable specification prevented a serious bug from reaching the pro-duction environment. A few weeks after the executable specification was created a refactoring of common code made the legacy system behave incorrectly. This was caught by the executable specification and the bug was taken care of before it could cause any damage. The interest payed on the business behaviour debt before the amortization was made include the damage that the bug would have caused.

(27)

7.2.3 Inefficient business debt

When knowledge crunching the scenarios of the legacy system problems with its current domain model were discovered:

• There was one redundant concept. The concept was used heavily in the legacy system but it was very technical in nature and did not have any counterpart in the business domain. The non-technical stakeholders pre-ferred to use another concept.

• There was one missing date-concept. In the legacy system there were three different date concepts, hidden deep in the model as technical details. These dates did not make sense to the domain experts. Instead they preferred to use another date-concept which was missing in the current domain model.

• There were two missing event-concepts. The legacy system acted upon different events in the business domain but these events were not explicitly modeled in the current domain model but instead represented as technical details.

Measuring the model debt with theModelDebtOMeterafter the amortization yielded the following result:

The inefficient business debt is 4 since there are 4 inefficiencies in the current domain model (1 redundant concept and 3 missing concepts). In the desired domain model the redundant concept would be removed and the three missing concepts would be added.

Interest

The redundant concept has been the cause of many misunderstandings between the developers and non-technical stakeholders of the legacy system. The use of technical concepts instead of the three missing concepts has led to a system which is not understandable by non-technical stakeholders. This in turn has led to a legacy system which behaves incorrect at a regular basis.

7.3 Evaluation of the model debt concept

The model debt concept was evaluated by five non-technical stakeholders. The result of the evaluation, i.e. the average result of the quantifiable questions and a summary of the written comments, is described in the rest of this section.

7.3.1 Business isolation debt and business behaviour debt

The questions related to business isolation debt and business behaviour debt yielded the following average results:

(28)

I think thatbusiness isolation debt is a clear concept that is easy to un-derstand.

Agree Disagree

I think that business behaviour debt is a clear concept that is easy to understand.

Agree Disagree

The non-technical stakeholders thought that both concepts were clear and easy to understand. To make them even clearer they suggested that:

• More and better examples are presented.

• They are translated into Swedish.

7.3.2 Inefficient business debt

The question related to inefficient business debt yielded the following average result:

I think that inefficient business debt is a clear concept that is easy to understand.

Agree Disagree

The non-technical stakeholders did not agree nor disagree to the statement. To make inefficient business debt clearer they suggested that:

• The concept of a desired domain model is made clearer.

• It is better explained how inefficient model debt is measured.

• More and better examples are presented.

7.3.3 Model debt

The questions related to the model debt concept as a whole yielded the following average results:

I think that the relationship between the different parts of model debt

(business isolation debt,business behaviour debt,inefficient business debt) is clear and easy to understand.

Agree Disagree

I think thatmodel debt highlights relevant aspects of software quality.

(29)

The non-technical stakeholders thought that model debt highlighted relevant aspects of software quality and that the relationship between the different parts of model debt was clear and easy to understand. To make the relationship clearer they suggested that:

• More and better examples are presented. Especially the relationship be-tween business behaviour debt and inefficient business debt needs more clarification.

• Instead of having inefficient business debt as one part of the model debt concept, build the model debt concept around inefficient business debt.

• The model debt concepts are translated into Swedish.

7.3.4 ModelDebtOMeter

The questions related to the ModelDebtOMeter yielded the following average result:

I think that we should use theModelDebtOMeter to work with software quality at Avanza Bank.

Agree Disagree

The non-technical stakeholders think that theModelDebtOMetershould be used to work with software quality at Avanza Bank. They motivate this opinion by saying that the ModelDebtOMeter makes software quality understandable also for non-technical stakeholders, making software quality a common concern for both developers and non-technical stakeholders, resulting in a more efficient development process.

In comparison with the quality barometer the non-technical stakeholders think that:

• TheModelDebtOMeteris a more complete software quality model.

• TheModelDebtOMeterexplains and motivates why a value has been cho-sen for a software system in the quality barometer.

• TheModelDebtOMeterfeels more abstract and less business focused.

• It feels like working with theModelDebtOMeterwill require more time. The non-technical stakeholders think that theModelDebtOMeteris a good complement to the quality barometer.

(30)

Chapter 8

Discussion

The purpose of this thesis is to use Domain-Driven Design to concretize the technical debt metaphor so that software quality for a non-technical stakeholder becomes visible, measurable and understandable. The ModelDebtOMeter has made software quality visible by extracting and showing the domain model of a software system. The model debt concept has not only concretized the technical debt metaphor but also added measurability. The concept has been tested in a working software system and the results show that it behaves reasonably in practice. The interesting question is if model debt and the ModelDebtOMeter

has made software quality understandable for non-technical stakeholders.

8.1 Examples

Looking at the results of the evaluation, model debt as a whole is an understand-able concept. Business isolation debt and business behaviour debt was easy to understand even though the non-technical stakeholders thought that they would understand it better with more and better examples. Inefficient business debt was harder to understand but here the non-technical stakeholder also thought that the understandability would increase with more and better examples. The relationship between the different parts of model debt was also easy to un-derstand but, no surprise, to make it more unun-derstandable, the non-technical stakeholders wanted more and better examples. How should this demand for more and better examples be interpreted?

I think that this is an effect of the method used in this thesis. The model debt concepts were not introduced to the non-technical stakeholders until the evalua-tion was carried out giving them no more than a hour to digest the concepts. It was a difficult task to explain the model debt concept and theModelDebtOMeter

in such a short amount of time. More and better examples would most likely increase the understandability but I wonder how much? Perhaps a better ap-proach would be to modify the method?

8.2 The model debt concept

The inefficient model debt was not as easy to understand as the rest of the parts of the model debt concept. In particular the non-technical stakeholders thought

(31)

that it was difficult to understand the concept of a desired domain model and the way that inefficient business debt is measured. They also thought that the relation between business behaviour debt and inefficient business debt was unclear. One of the non-technical stakeholders suggested that the model debt concept should be rearranged and centered around inefficient business debt.

I definitely think that there is room for improvement of the model debt concept. The reactions of the non-technical stakeholders clearly shows that some part of the concept is not quite right. Perhaps the model debt concept should be rearranged?

8.3 Model debt and software quality

The non-technical stakeholders strongly agree that model debt highlights rele-vant aspects of software quality. When comparing theModelDebtOMeterto the quality barometer the non-technical stakeholders think that the former is a more complete software quality model but at the same time they think that it feels more abstract. This reveals something about the non-technical stakeholders view on software quality.

I think that the non-technical stakeholders want software quality to be some-thing easy. They don’t want to be involved in the real complexities that exist on any software project. At the same time they realize that they need some involvement to be able to understand software quality at any depth. Clearly, the challenge is to strike a balance between making software quality easy to explain and preserving its key complexities.

This discussion highlights an important aspect about communicating soft-ware quality to non-technical stakeholders: the simplification of softsoft-ware quality. The model debt concept focuses on the quality of the domain model. There are many other important aspects of software quality such as performance and the quality of the technical infrastructure that are not included in the concept. I don’t think that it is possible to bridge the software quality gap without exclud-ing some parts of software quality while focusexclud-ing on others.

8.4 The

ModelDebtOMeter

During the evaluation, the ModelDebtOMeter was used to explain the model debt concept to the non-technical stakeholders by showing how the concept applied to the legacy system. Therefore, the understandability of the model debt concept is related to the quality of theModelDebtOMeter. Not much effort has been put into making the ModelDebtOMeter easy to understand for non-technical stakeholders.

I think that the software quality gap could be reduced further by improving theModelDebtOMeter. It is likely that this could also reduce the need of more and better examples.

8.5 Model debt and Domain-Driven Design

Eric Evans describes DDD as a set of driving principles used when developing software [1]:

(32)

• Speak aubiquitous language within an explicitly bounded context.

• Explore models in a creative collaboration of domain practitioners and software practitioners.

• Focus on thecore domain (the part of the domain model which matters most to the business).

Of these three driving principles the model debt concept captures the first two of them. The third one is not dealt with at all. It didn’t make sense to include the concept of core domain before knowing if the approach taken by this thesis was at all successful. For the same reason, DDD-concepts which describe the domain model at a more detailed level such as entity, value object, factory, repository, service and aggregate are not included in the model debt concept.

Given the successful results I think that the model debt concept could benefit from the inclusion of more DDD-concepts.

8.6 Conclusion

Model debt reduces the software quality gap between developers and non-technical stakeholders. I think that the gap could be reduced further by:

• Involving the non-technical stakeholders more in the model debt con-cept. This can be achieved by amortizing model debt while continu-ously discussing the results with the non-technical stakeholders using the

ModelDebtOMeter.

• Rearranging the model debt concept so that it becomes easier to under-stand.

• Including the DDD-concept of core domain into the model debt concept. Doing so would make it possible not only to communicate the size of the model debt but also to communicate which parts of the debt that should be amortized.

• Including DDD-concepts which describe the domain model at a more de-tailed level. This would make it possible to have much richer discussions about the quality of the domain model together with the non-technical stakeholders.

(33)

References

[1] Gojko Adzic. Eric Evans: Domain driven design redefined.http://gojko. net/2010/06/11/eric-evans-domain-driven-design-redefined/. [2] Gojko Adzic. Bridging the Communication Gap. Neuri, 2009.

[3] Nanette Brown, Yuanfang Cai, Yuepu Guo, Rick Kazman, Miryung Kim, Philippe Kruchten, Erin Lim, Alan MacCormack, Robert Nord, Ipek Ozkaya, Raghvinder Sangwan, Carolyn Seaman, Kevin Sullivan, and Nico Zazworka. Managing technical debt in software-reliant systems.FoSER ’10 Proceedings of the FSE/SDP workshop on Future of software engineering research Pages 47-52, 2010.

[4] W. Cunningham. The WyCash Portfolio Management System. http: //c2.com/doc/oopsla92.html.

[5] W. Cunningham. Ward Explains Debt Metaphor. http://c2.com/cgi/ wiki?WardExplainsDebtMetaphor.

[6] Eric Evans. Model Exploration Whirlpool. http://domainlanguage. com/ddd/whirlpool/Domain\_Language\_Model\_Exploration\

_Whirlpool\_v2010-06-19.pdf.

[7] Eric Evans. Domain-Driven Design. Addison-Wesley, 2003.

[8] M. Fowler. TechnicalDebtQuadrant. http://martinfowler.com/bliki/ TechnicalDebtQuadrant.html.

[9] Robert C. Martin. A Mess is not a Technical Debt.

http://blog.objectmentor.com/articles/2009/09/22/ a-mess-is-not-a-technical-debt.

[10] S. McConnell. Technical Debt. http://blogs.construx.com/blogs/ stevemcc/archive/2007/11/01/technical-debt-2.aspx.

(34)

Appendix A

Evaluation Form

1. I think that business isolation debt is a clear concept that is easy to un-derstand.

Agree Disagree

What can make the concept clearer?

2. I think that business behaviour debt is a clear concept that is easy to understand.

Agree Disagree

3. I think that inefficient business debt is a clear concept that is easy to understand.

Agree Disagree

4. I think that the relationship between the different parts of model debt

(business isolation debt,business behaviour debt,inefficient business debt) is clear and easy to understand.

Agree Disagree

(35)

5. I think that model debt highlights relevant aspects of software quality.

Agree Disagree

Which aspects of software quality is missing?

6. Compare the software quality of the legacy system as estimated by the

quality barometer with the software quality as estimated by the

ModelDebtOMeter. What are the pros and cons of using the

ModelDebtOMeter?

7. I think that we should use the ModelDebtOMeter to work with software quality at Avanza Bank.

Agree Disagree

(36)

Appendix B

ModelDebtOMeter

Screenshot

The ModelDebtOMeter showing the domain model of the legacy system at Avanza Bank and its model debt after all amortizations have been performed. All model blocks have been renamed to ModelBlock

Bridging the software quality gap