Reaching Software Development Maturity with Continuous Delivery. Fredrik Stål Viktor Ekholm Faculty of Engineering, Lund University

(1)

Reaching Software Development

Maturity with Continuous

Delivery

Fredrik St˚

al

Viktor Ekholm

Faculty of Engineering, Lund University

(2)

(3)

Abstract

In the area of software configuration management a configuration manager works with almost every part in the software development process. The responsibility is to ensure efficiency at the highest possible level for a development team, which is simplified a great deal with a mature and well defined development process.

The purpose of this thesis was to analyze and locate key areas in a software development process for small teams and define methods of improvement that will serve as goals for to reach process maturity. The work is mostly based upon the concept known as continuous delivery, which aims at maintaining a releasable product after each software code integration. The project was carried out on a consulting firm called Softhouse that also has in-house development projects with mobile applications in teams ranging from one to seven developers.

The work started with interviewing the developers with the goal of finding the root-cause of the current process, as well as identifying both the possibilities of improvement and determine the requirements they had on their own work. To narrow the scope we focused on working together with a project that started and had its deadline during this thesis. That project was split into three smaller sub-projects, which gave us the chance to explore how more than one development process was normally carried out and how collaboration was done.

Together with the developers in one of the sub-projects we designed and implemented a new environment with the aim of solving both the perceived and real problems that were discovered during analysis. The purpose of the environment was to perform manual tasks by automation and introduce the starting points of continuous delivery. Measurements on the process was done before and after the improvement to find what benefits gained and any eventual costs. The results of the measurements were positive and although the change was a very small one the benefits was still detectable.

Based on our analysis and results from measurements we developed a maturity model. The primary objective of the model is define a set of goals to improve a software process. The model is split up into different levels of maturity and to reach a higher level the process first has to achieve the goals for the next maturity level. This maturity model is the result of our master’s thesis and is still in its experimental stages. It would require further testing and research to be fully developed.

Keywords

Software development maturity, software process improvement, software configuration manage-ment, continuous delivery, continuous integration, automation, agile development

(4)

(5)

Preface

Configuration management has had many definitions over the years. One of the older definitions tells us that the goal of configuration management is to “maximize productivity by minimizing mistakes”. Another definition is that configuration management “represents general techniques for controlling the creation and evolution of object”. As its name applies configuration management manages configurations and a configuration can been seen as an image of an object at a certain point in time. As the amount of variations increases for the components of the object, so does the number of available configurations. Without sufficient ways of maintaining all these configurations, their history will be lost and without history our mistakes will repeat themselves.

This thesis is targeted for individuals with some expertise within the subject of software devel-opment, be it developers, project managers, configuration managers or undergraduate students in the field of computer science. We will attempt to keep our discussions at a level so that any person belonging to the aforementioned groups can make use of the information presented and discussed in this report.

(8)

(9)

Acknowledgments

We would like to thank our supervisor, Christian Pendleton, and our examiner, Lars Bendix, for their invaluable guidance and great engagement and interest throughout our work. We would also like to thank all employees of Softhouse in Malm¨o, the company where we carried out the work you are about to read, for their help and hospitality.

(10)

(11)

Chapter 1

Introduction

As fine spirits mature and develop a more complex and interesting taste over the years, the same can, almost, be said about software development. Having a mature software development process may not produce any extraordinary taste sensations, but it does give you a comfortable and effective work process. The prerequisites are to put in effort to actually mature the software process.

In short, improving your software process aims to rid you of any manual, repetitive processes and possible downtimes due to events you have to wait for to complete. Unnecessary rework is considered an anti-pattern and a very bad practice. Automating this hard labor intends to give developers a break from tedious, manual tasks, but in theory quality and consistency would improve, since there is less chance of careless mistakes due to automation.

A fundamental building block in software process improvement is the practice of continuous de-livery. Through this practice we expect less integration problems as software code is integrated in small portions several times per day. Through automated test executions upon integration, we receive iterative feedback which tells the status of our software.

1.1 Problem Statement

Softhouse is first and foremost a consulting firm but also conduct some in-house development. These development projects became our focus and it was initially perceived that the in-house developers were in need of a common build server.

Through interviews with developers and stakeholders, our take on the in-house environment was that the nature of the company required developers to be given assignments on other companies between projects. Their knowledge of processes is then temporarily lost. As there is no collective knowledge of processes, there is a lot of work made ad-hoc at project start. In a way this has a relation to open-source projects, where it is assumed that contributing developers will not contribute during the entire lifetime of the project.

Another problem is that no value is seen in taking the required time to mature and baseline processes. Work hours are considered to be billable and the customer is most certainly not interested in paying good money for advancing the processes of Softhouse after they already accepted them as developers. The interest for Softhouse in developing more efficient processes is to place more time and focus on developing the software and thus increase productivity.

(12)

2 Chapter 1. Introduction

1.2 Purpose

The purpose of this thesis was to, through interviews and analysis, locate key areas in the develop-ment process that could benefit the most from improvedevelop-ment. Although being a smaller company, Softhouse had multiple projects running simultaneously with different number of developers in each one, ranging from one to six. Our first target was only the projects that developed applica-tions for the Android platform and try to extend the process by adding additional features and tools. The intention was to improve, not only to increase efficiency of the development, but also the communication with customers.

After a certain amount of weeks we changed our target to a newly started project that had three smaller parts. These parts consisted of an ASP.NET website, an Android and an iPhone application, respectively. The motivation of this change of target was to have the opportunity of accessing and monitoring a live project taking shape in real-time, and not just basing our work on theoretical or old projects. Also, the diversity aspect of this three-parted project interested us. The primary goal then became to analyze the process used for each part and together with the developers work with improving beneficial areas during the course of the project. It was discovered that one of the smaller project contained a deployment procedure that was performed several times a day to a testing environment, which we saw as a perfect opportunity to study how processes are carried out on Softhouse.

There was no unified or general approach for projects, which had led to that developers on each part of the studied project had performed implementations on an ad-hoc basis. Besides analyzing the reasons and root-causes of the chosen methods we also wanted to implement an environment with the purpose of introducing automated procedures for as many of the manual activities as possible. This would further benefit our own work, but our initial hypothesis was that it would also help to increase knowledge of well defined practices and make the company see the benefits of such process improvements from a practical perspective.

1.3

Chapter 2

The Domain [3]

In this chapter we will outline the methodology to define the domain, which is the focus of further analysis. The final two sections will compare three different types of projects that we studied to each other. This chapter will also discuss the initial problems we perceived while exploring the domain.

2.1 Definition [1.5]

Before making an attempt at analyzing a project, we would need to define our context, so as to know our boundaries. The context will give a basis for our analysis, as the analysis is dependent on the context. The context specifies what we need to look for when doing the analysis. Without knowing the context there is no way of knowing what kind of data we need to collect to make a more detailed analysis of the perceived problems.

• Company context

– Available resources

– Company size

– Competence levels

– Knowledge of used tools

• Process context

– Process work model (agile, waterfall, Lean, etc.)

– Practices followed

– Patterns and anti-patterns

• Project context

– Project type

– Project size

– Personality types (attitude problems)

(14)

4 Chapter 2. The Domain [3]

The motivation for why these divisions exists is related to questions that can be asked during interviews with different types of key personnel.

Below is our list of defined data items that we collected through interviews to define the domain.

Company size. Competence levels. Knowledge of used tools. Process work model. Practices followed. Project type. Project size.

Team setup (activity distribution).

2.2 Project Types [0.5]

The results from our definition of the domain/context was that we had a project divided into three smaller projects. Following is a list of descriptions for the different project types.

ASP.NET website server application. A website application using Microsoft’s .NET frame-work. In this method there is no distinction between build artifacts and source code, since compilation is done at run-time. Locked to Windows platform.

Android client application. Can be executed on emulator or actual hardware. Similar to reg-ular Java applications.

iPhone client application. Uses binary artifacts. Locked to OS X platform. Running these kinds of applications during development requires that the cell phone is explicitly granted access in the source code.

We will take up differences between the project types and how they integrate in the project scope.

2.3 Perceived Problems [1]

During our initial analysis of the project types, we were able to define some initial, perceived problems. These problems are directly linked to the context of this thesis which will be motivated by a discussion of their impact on the three types of contexts defined in the first section in this chapter.

Perceived problems for ASP.NET project:

1. Tests are not part of the project, they are a project of their own

2. One developer is only writing unit test while the other is only writing black-box tests 3. The test environment is not identical to the production environment

4. Builds are performed on development environment and then again of test/production envi-ronment

5. Source code on both development and test environment 6. The server build could cause failures in client tests

(15)

2.3 Perceived Problems [1] 5

7. Changes to the database are done on the test server and are not recorded in any way Perceived problems for Android project:

1. Workspace-specific files are checked in to the repository

2. Libraries, upon which the application is dependent, are not checked in to the repository 3. Unit tests is on another repository than the code

4. No acceptance tests are written and done completely manual 5. Test results on production-like environment are not documented 6. Production-like environments are required for all tests.

(16)

(17)

Chapter 3

Locating the Gremlin [5]

The problem analysis main goal was to discover the root-cause to the initial, perceived problems that are defined in chapter 2.

This chapter will detail how we performed the analysis to later arrive on a set of requirement we placed on arbitrary solutions that are discussed later in chapter 5. Although we did separate analysis for each of the sub-projects, this chapter will discuss an abstract approach to a root-cause analysis on a project with inclusions to our own studies.

3.1 Preliminary Investigation [1/2]

Since the analyzed project was divided into three smaller projects it was decided to treat each “sub-project” by as a project of its own as a starting point.

3.2 Define the Project Type [1]

Outline a definition of a general project as well as the three project types encountered during the project.

3.3 Analyze Current Process [2]

Describe the analysis performed to define the current process and what was discovered.

3.4 Discover the Real Problems [3/4]

Explain what kind of (real) problems the root-cause analysis found and if some of them are shared between project types.

(18)

8 Chapter 3. Locating the Gremlin [5]

3.5 Define Requirements [1/4]

Argue for a set of requirements that are based on the defined problems in the previous section and takes the context into consideration.

(19)

Chapter 4

The Cornerstone [7]

Discuss the impact of software development maturity and continuous delivery on a software process analysed in chapter ??. Define any requirements and constraints placed on a development team when implementing solutions based on the two concepts. Argue the relations to advantages and drawbacks as well as form a discussion that will introduce the designing of solutions.

4.1 Software Development Maturity [2]

Argue for the benefits of reaching higher levels of software development maturity and why a company should spend time and effort to improve their process.

4.2 Continuous Delivery [4]

Introduce the concepts and practices behind continuous delivery and outline the benefits and potential drawbacks it gives to software development.

4.2.1 Continuous Integration [1]

Lay the foundation for continuous delivery by describing the process of continuous integration, according to Beck & co.

4.2.2 Continuous Deployment [2]

Discuss how the concept of continuous deployment can be used together with a higher level of maturity.

4.2.3 The Deployment Pipeline [3/4]

Argue for the purpose with a build pipeline and the reason it is used together with continuous delivery.

(20)

10 Chapter 4. The Cornerstone [7]

4.3 Patterns and Anti-Patterns [1/2]

(21)

Chapter 5

Designing the Solution [3]

Given the “requirements” from chapter 3, you discuss and argue the possible solutions to each problem and discuss advantages and drawbacks of each solution. Given your context you will be able to argue why you chose one solution over implementations and results from previous studies.

5.1 Continuous Integration Server [1/2]

Discuss solutions for CI servers and the pros and cons with different solutions.

5.2 Development Environments [1/2]

Design of solutions that change the nature of a development environment and has connections to types of platforms. Example: virtual machine, optimal CI server platform.

5.3 Branching Strategies [1/2]

Discuss different branching strategies.

5.4 Distributed Development [1/4]

Discuss possible approaches to tackle distributed development.

5.5 Survival of the Fittest [1/2]

(22)

(23)

Chapter 6

Implementation and Its

Measurements [8]

Document the results and measurements from the selected method/solution. Describe how it was implemented into the context, potential roadblocks that was encountered and discuss how they could have been avoided.

6.1 Implementations [5]

Detail the implmentation part for the three projects.

6.1.1 ASP.NET Website Project [2]

Document the implementation for the ASP.NET project.

6.1.2 Android Project [2]

Document the implementation for the Android project.

6.1.3 iPhone Project [1]

Document the (non-existent) implementation for the iPhone project.

6.2 Measurements [2]

Discuss the results of measurements before and after implementation, etc., and discuss initial conclusions.

(24)

(25)

Chapter 7

Evaluation [5]

Evaluate and discuss the “validity” of your work. Are there any “restrictions” or limitations on how/where your results/solutions work? Discuss and relate to other work in the same area. Possibly something about future work to extend/complete your work.

7.1 Validating Implementations [2]

Validate the implementations that was discussed in previous chapter.

7.2 Limitations [1/2]

Discuss what the limitations/restrictions of our solution are.

7.3 Related Work [1]

Compare the results of this thesis with those from other, related studies.

7.4 Further Studies [1/2]

Bring up areas of our thesis we have found need further studies to be considered somewhat com-plete; areas which we haven’t had the time to pursue fully or areas considered to be out of scope for the thesis.

(26)

(27)

Chapter 8

Conclusions

The work that has been conducted during this thesis gave us a lot of ideas, some of them which we were able to fulfill with implementations. The majority of results gave positive impressions to continuous delivery. No one can deny that reaching high levels of software development maturity requires effort from the majority of the organization, but there is no particular reason as to why this shouldn’t be one of the most prioritized goals.

8.1 Results

The most promising result from this master’s thesis has been that even minor changes, which requires very little effort, have significant effects and benefits on a software process. Although, implementing a solution for one of the parts of the studied project proved only to be a challenge for reasons related to installations and first-time setup. Since neither of us have had much experience in this type of work, this resulted in unexpected problems.

We saw that the changes we introduced and have discussed will be very beneficial for a software company, especially in the long run. The results have also shown that a developer will increase his or hers interest in their own work, which is related to the fact that a process improvement is targeting just that. Given time a changed process will also become a natural way of working, which eventually leads to further improvement and higher levels of maturity.

There are still some issues that needs to be tackled linked to the view of concepts, such as con-tinuous delivery and the tools developers can exploit to simplify activities. One of the most common issues we ran into during the project was a misguided view that tools are something of a wizard or a magic wand. The larger application, for example a continuous integration server application, is supposed to magically solve all your problems while you, the developer, only have to install the application and tell (configure) it what to do. The magic wand consists of tools like your version-control system that you somehow wave over your software code and every fault is corrected.

Another result of our thesis is that it has to, at least in some way, be defined who is responsible for maintaining the different environments. We have seen how the natural approach of everyone’s responsibility quickly leads to no one’s responsibility. In the same way, if too few employees are responsible then the question and the problem will still be present should they leave. As mentioned before in this report, a consulting firm with in-house development has the same requirements for developers as an open-soruce environment and thus needs to have the same flexibility.

The above mentioned issues are what we found in our context, taking another context into con-sideration might have a lot more issues or perhaps fewer. Our work showed that issues can not

(28)

18 Chapter 8. Conclusions

simply be abstracted into a general solution, but be solved differently from company to company. To reduce the friction in a software process improvement and remove ad-hoc procedures a ma-turity model, like the one we developed, will narrow the scope for key areas and help to create requirements to reach higher levels of maturity.

It proved very well to combine the principles of continuous delivery together with that of software development maturity. The end result became a model that is incremental by nature and more related to development activities and therefor more easily interpretable for developers. Although faster releases is a given benefit for continuous delivery this thesis has shown that the concept is much more than that. The core feature, the build pipeline, has shown to be a well-designed approach when implementing control and automation of your software.

8.2 Thoughts for the Future

As a final note it is worth taking the future into consideration. The solutions presented here in this thesis might seem useless or no longer feasible in, for example, ten years. However we can, with confidence, say that the problems related to software development that are the main focus for a process improvement will still be and forever be present. The impact they have on development might differ, but the root-causes will live on and need to be maintained.

(29)

Bibliography

[ARH06] Simon Alexandre, Alain Renault, and Naji Habra. OWPL: A Gradual Approach for Software Process Improvement in SMEs. In Software Engineering and Advanced Applications, 2006. SEAA’06. 32nd EUROMICRO Conference on, pages 328–335. IEEE, 2006.

[BA04] Kent Beck and Cynthia Andres. Extreme Programming Explained: Embrace Change. Addison-Wesley Professional, 2004.

[Bab86] W.A. Babich. Software Configuration Management: Coordination for Team Produc-tivity. Addison-Wesley, 1986.

[Bec99] Kent Beck. Embracing Change with Extreme Programming. Computer, 32(10):70–77, 1999.

[Bur03] Ilene Burnstein. Practical Software Testing: A Process-Oriented Approach. Springer-Verlag New York Incorporated, 2003.

[Dar00] S. Dart.Configuration Management: The Missing Link in Web Engineering, chapter 3. Computing Library. Artech House, 2000.

[FF06] M. Fowler and M. Foemmel. Continuous Integration. ThoughtWorks, 2006.

[FH01] M. Fowler and J. Highsmith. The agile manifesto. Software Development, 9(8):28–35, 2001.

[Gei12] M. Geiss. Continuous Integration and Testing for Android, 2012.

[GM01] Athula Ginige and San Murugesan. Web Engineering: An introduction. Multimedia, IEEE, 8(1):14–18, 2001.

[Hem13] P-G Hembrink, J. Stenberg. Continuous Integration with Jenkins, 2013.

[HF10] J. Humble and D. Farley. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Addison-Wesley, 2010.

[HRN06] Jez Humble, Chris Read, and Dan North. The Deployment Production Line. In Pro-ceedings of the Conference on AGILE 2006, AGILE ’06, pages 113–118, Washington, DC, USA, 2006. IEEE Computer Society.

[KWBF07] Michael Karlesky, Greg Williams, William Bereza, and Matt Fletcher. Mocking the Embedded World: Test-Driven Development, Continuous Integration, and Design Pat-terns. InProc. Emb. Systems Conf, CA, USA, 2007.

[Lar13] L Larsson, D. Gustafson. Automatiserad release som en del av var story, 2013. [MDHG01] San Murugesan, Yogesh Deshpande, Steve Hansen, and Athula Ginige. Web

Engineer-ing: A New Discipline for Development of Web-Based Systems. InWeb Engineering, pages 3–13. Springer, 2001.

(30)

20 BIBLIOGRAPHY

[Mil08] A. Miller. A Hundred Days of Continuous Integration. In Agile 2008 Conference, pages 289–293. IEEE, 2008.

[Mor10] Mario E. Moreira. Adapting Configuration Management for Agile Teams: Balancing Sustainability and Speed. Wiley, 2010.

[PCCW93] Mark C Paulk, Bill Curtis, Mary Beth Chrissis, and Charles V Weber. Capability Maturity Model, Version 1.1. Software, IEEE, 10(4):18–27, 1993.

[Pra] Praqma. Software Configuration Management. Praqma A/S, Allerød Stationsvej 4, DK-3450 Allerød, Denmark.

[Sho06] James Shore. Continuous Integration on a Dollar a Day. http://www.jamesshore. com/Blog/Continuous-Integration-on-a-Dollar-a-Day.html, 2006.

[Urb12] UrbanCode. ROI: The Value of Deployment Automation. http://www.urbancode. com/html/resources/white-papers/The_Value_of_Deployment_Automation/, UrbanCode, Inc., 1228 Euclid Ave., Suite 855, Cleveland, OH 44115, 2012.

(31)

Reaching Software Development Maturity with Continuous Delivery. Fredrik Stål Viktor Ekholm Faculty of Engineering, Lund University