CONTINUOUS INTEGRATION

In the following pages we will discuss the policies and systems that together make up the process called Continuous Integration. This document is written for IT managment who are looking

CONTINUOUS

INTEGRATION

REALISING ROI

IN SOFTWARE

DEVELOPMENT

PROJECTS

Continuous Integration

Executive Summary

Introduction to Continuous Integration The Systems

Source Code Control Unit Testing Code Validation Documentation Integration Testing Regression Testing The Tools Subversion PHPUnit CruiseControl/phpUnderControl PHP_CodeSniffer phpDocumentor Phing Return on Investment Conclusion

Table of Contents

The main concept behind Continuous Integration as described by Martin Fowler is simple: “Every developer checks in their code, every day.” Adopting this policy reduces the time it takes to integrate a developer’s code into the existing codebase to almost zero. Regular and frequent check-ins also allow testing systems to be developed that will regularly scan the entire code base for problems and raise red flags early in the process rather than later, when the bugs are more expensive to fix. These benefits allow teams to improve the quality of the software they deliver. Because developers are finding bugs earlier in the process rather than later, the cost of developing internal software is reduced, thus increasing the ROI of the team.

When teams adopt this policy, systems can be put in place to help make sure that the code being written integrates with other developers’ code and the existing codebase. This methodology, along with the accompanying support systems, is known as a Continuous Integration system.

From the system perspective, a stable and useful Continuous Integration system can be assembled using several best of breed, open source applications based on PHP.

Executive Summary

Continuous Integration

• Subversion for Source Code Control

• phpUnderControl/Cruise Control for Integration and regression testing 4

php Documentor documentation • Code_Sniffer for code validation • Phing for build tasks

Combining these systems together, teams can integrate their code changes quickly, build on a scheduled basis and ensure that the system is not only working according to their understanding of what is to be built, but working according to the client’s specifications.

Software development continues to grow increasingly more complex. Even when dealing with dynamic languages like PHP, the systems being built today dwarf the systems of even five years ago. Many development teams however have not upgraded their development process and systems to meet the new demands. Today’s teams must be well versed in the best practices of software development like source code control, unit testing, integration testing and documentation. These practices, when coupled together, form the basis of a Continuous Integration system.

Continuous Integration is a term coined by Martin Fowler in a paper by the same name.

Introduction to Continuous Integration

Continuous Integration

I

When programming in today’s dynamic languages, the concept of a “build” does not mean the same thing as it does for traditional compiled languages. However the other concepts he discusses still have a place in software development. Continuously integrating changes into the main code base so that their effects can be quickly seen – and fixed if necessary – is an important step in controlling the cost of bugs and the quality of your overall system.

As discussed earlier, Continuous Integration is a series of inter-related services that, when used together, accomplish the goals behind Mr. Fowler’s concept of Continuous Integration. Those goals are to eliminate the Integration phase of any software project and to identify bugs earlier rather than later.

“Continuous Integration is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including

Continuous Integration

The services used in Continuous Integration are: • Source Code Control

• Unit Testing • Code Validation • Documentation • Integration testing

• Regression testing (builds)

All of these systems can be setup independently and used ad hoc. However, forward think-ing teams are now integratthink-ing all of them to help identify and find fix small problems early rather than large problems late.

The underlying principle of Continuous Integration, frequent integration of code being written, is handled by the source code control system. The secondary goals, unit testing, code validation, documentation and integration testing, all revolve around this system.

The Systems

Continuous Integration

Source Code Control

As stated, Source Code Control is the heart of any Continuous Integration system. Source Code Control is the main repository for the code and the first line of defense in many Continuous Integration systems. Aside from serving as the focal point for your source code and a backup in case a developer’s machine breaks down, your source code control system provides you with important hooks that can be used to automate pieces like unit testing.

Unit Testing

Many teams use a two pronged approach for Unit testing. The first prong is the developer running the tests manually. This type of testing is helpful in creating the code but relies on the developer to remember to run the tests before check-in.

The second prong addresses this shortcoming by automating unit testing and reporting. Many teams use hooks in programs like Subversion to automatically run Unit Test suites. There are two schools of thought when doing this. The first is to deny check-in if the code fails its unit testing. The other is that unit tests are run on check-in but the check-in can proceed but the error is reported. Either way, when bugs are introduced into the system, developers know about it immediately and can fix them quickly.

Code Validation

It is often said that the wonderful thing about standards is that there are so many to choose from. Nowhere is this more evident than with PHP coding standards; there is an embarrassment of riches when choosing a standard for your team. The problem, though, is not choosing one but enforcing it once chosen. Once you and your team have settled on a standard, a Continuous Integration system can help enforce it by routinely scanning the repository and identifying code snippets that do not match the agreed upon specification.

While nothing can replace well written end-user documentation, good API documentation is invaluable to developers. This is never truer than when new developers are moving on to a project or picking one up after having spent time away from it working on other projects. Many languages have adopted the JavaDoc style for API documentation. This method allows for the documentation to be read in-line by the developers while at the same time, can be parsed by appropriate systems to automatically generate a comprehensive set of docs for the project.

Continuous Integration

Documentation

Integration Testing

The concept of software builds does not translate well into dynamic language develop-ment. Since all code is parsed and tokenized at runtime, the traditional processes of compiling and linking do not come into play. The routine of checkout, compile and linking is replaced by Integration Testing. Unit testing is an integral part of any software develop-ment project and is the best way to find bugs in your code early. Even the most rigorous unit testing cannot guarantee that all the pieces in the system will work together. For this you need to implement an Integration Testing regime. Integration testing is fitting all the individual pieces together and making sure that they work together. Integration testing can take three forms.

By far the most popular methodology of Integration testing is bottom up testing. In Bottom Up testing the first step is to re-run all of your unit tests. Once they have been re- tested in full, you use a system like Selenium to test how the system as a whole works. Those reports are posted online for all to see.

• Bottom Up

Using the Top Down methodology, all of the units are assembled into a system and then mod-ules are tested. Below the modmod-ules, individual units are tested.

Look for mindshare

Continuous Integration

Test plans for integration testing are important Teams need to make sure that they are testing what the client will be testing in the Acceptance testing phase.

Regression Testing

Once a milestone has been achieved, the test plans for the features in that milestone are integrated into the “Regression Testing” testing plan. Regression testing makes sure that once something is working, it stays working. It’s not enough to simply test the features that you are currently working on, you have to continue to test completed features to make sure that new code does not have adverse affects.

“Also as a consequence of the introduction of new bugs, program maintenance requires far more system testing per statement written than any other programming. Theoretically, after each fix one must run the entire batch of test cases previously run against the system, to ensure that it has not been damaged in an obscure way. In practice such regression testing

must indeed approximate this theoretical idea, and it is very costly” -Fred Brooks, The Mythical Man Month

Now that we’ve discussed what a Continuous Integration System consists of, let’s talk a bit about how to put one together. We will discuss the best of breed tools for PHP-based shops to use to implement Continuous Integration. While we will discuss the tools here, we will not go into the actual setup and integration of these tools. There is an excellent primer on that very subject on Inviqa’s techPortal site titled “Getting Started with phpUnderControl i ” by Marc Veldman.

The Tools

Continuous Integration

The undisputed leader in Source Code Control for open source projects is Subversion. While the venerable CVS is still widely used, most teams have a migration plan already to move to Subversion. Since your Source Code Control system will be the heart of your Continuous Integration system it is recommended that that you get this important piece up and running before you move on to the next steps.

Subversion

PHPUnit

Unit testing, whether done at check-in or only during scheduled builds is the cornerstone of the Continuous Integration system. PHP has several good Unit testing frameworks to choose from; however, the most complete and the most widely accepted is the PHPUnit testing framework. Used by itself, it is a powerful tool for ensuring the quality of a code base. When used as an integral part of a Continuous Integration system it helps improve the ROI of the project by finding problems earlier rather than later.

Continuous Integration

CruiseControl/ phpUnderControl

The heart of any Continuous Integration system is the continuous build server. In the system being described CruiseControl and the PHP plugin for it, phpUnderControl, serve this role. CruiseControl is one of the most popular Continuous Build tools available and while it is written in Java there are builders and plugins available for many different languages, including Phing for PHP. phpUnderControl is the add-on application which allows for the integration of other PHP tools.

PHP_CodeSniffer

By hooking PHP_CodeSniffer into the Continuous Integration system, you can automate the enforcement of your accepted coding standard. PHP_CodeSniffer will analyze the code in your project and report discrepancies. The report generated can be used to identify problems and resolve them quickly. Maintaining strict adherence to a coding standard will pay for itself in the maintenance phase of your project.

phpDocumentor

phpDocumentor is the accepted standard tool for building API documentation for PHP projects. Not only can it generate nicely formatted documentation, most modern IDEs support it, allowing for code completion on user code in addition to native language calls.

Phing

All build systems need a build utility. Unix/Linux has had make for many years now, Java has Ant and .Net has Nant and more recently MSBuild. PHP’s build tool is Phing. Phing is based on the concepts of Apache’s Ant and allows you to complete any task you would be able to using Ant, make or any other build system. CruiseControl has plugins that support using Phing.

Any project undertaken by a company has to show that it is valuable to the company. When dealing with IT, this value, or the Return on Investment, is usually measured in either dollars contributed back to the bottom line or in time saved. Continuous Integration systems add value to the development process and the project as a whole and their value can be measured using both metrics. Overall, when properly installed, configured, and used, a Continuous Integration system will allow your teams to build projects faster and cheaper.

The first, and arguably the most visible, benefit of Continuous integration is that it allows teams to identify and fix bugs early in the development cycle, as they appear. Conventional IT wisdom tells managers that the earlier you can find a bug, the cheaper it is to fix it. Since it’s not yet possible to fix bugs as the programmer makes them, the next best thing is to find and report them when the code is checked in. This way the problem is fresh on the developer’s mind and fixing it doesn’t derail any other part of the project. Fixing bugs quickly, thus cheaply, means that systems are deployed with fewer bugs, and because expensive late- in-lifecycle debugging is not necessary, it allows the project to be delivered more quickly and within budget.

Return on Investment

Continuous Integration

The second and less visible improvement that Continuous Integration brings to projects is oversight. PHPUnit, CruiseControl, and phpUnderControl all bring with them published reports that can

.... when properly installed, configured and used, a Continuous Integration system will allow your teams to build projects faster and cheaper

are useful for determining the health of the project. Project managers can use this

information to make decisions on how to proceed with the project. Just like with the health of a human body, problems found early can be corrected more easily than those found later on.

Continuous Integration systems require an upfront investment in time to install and configure. Thought needs to be given as to where the systems will physically reside. Once the initial investment has been made, however, all projects utilizing them can enjoy their benefits with only marginal costs involved in project setup and overall system maintenance. The latter can be distributed across all the projects using the system.

Continuous Integration

References

Conclusion

This whitepaper has discussed the systems that make up a Continuous Integration sys-tem, made specific recommendations on which systems to use, and shown the ROI of using a Continuous Integration System. However, no system can make a difference if your team does not believe in them and use them. Implementing a Continuous Integration system is more about team attitude than the systems you deploy. The services we have described here can help teams identify bugs early but only if the system is used. The cen-tral tenant of Continuous Integration, “Every developer checks in their code every day”, cannot be enforced by a system, it has to be an attitude adopted by not only each devel-oper but the team as a whole.

As has been shown, however, the Return On Investment for deploying a Continuous In-tegration system can be measured in both the short term and long term. In the short term, bugs are cheaper to fix when they are found early in the development lifecycle. In the long term, as projects mature, the continuity of the code base will be maintained as different developers migrate in and out of the project. Continuous Integration and the systems that support it will keep your projects clean, tested and ready for deployment at a moment’s notice.

i http://techportal.inviqa.com/2009/03/03/getting-started-with-phpunder-control ii http://subversion.tigris.org/ iii http://phpunit.de iv http://phpundercontrol.org/about.html v http://pear.php.net/package/PHP_CodeSniffer vi http://www.phpdoc.org/ vii http://phing.info/