Continuous Integration Comes to China.

(1)

Continuous

Integration Comes

(2)

Agenda

Time Topic Presenter

2:00 Introduction Tracy Shi

Emdoor Technology 2:15 Continuous Integration Anders Wallgren,

Electric Cloud

(3)

Continuous Integration

Anders Wallgren CTO, Electric Cloud

(4)

Integration Hell

Teams spend weeks or longer working

separately

Developer changes are infrequently built into the trunk

Code bases and build environments diverge

At some point all changes are integrated

Teams spend weeks or longer verifying that applications work

Time spent ‘integrating’ is tortuous, leads to integration ‘storms’ of broken builds

Developers have to track down bugs introduced months earlier

(5)

What is needed

Frequent integration builds (multiple times per day)

Find build issues earlier

Smaller changes mean troubleshooting is easier

Quicker feedback means you address problems when they are fresh

A common, shared build environment for all kinds of builds

Environment settings are common Reduces localized errors

Speed

Fast compile/link times Parallel build job steps

(6)

The Solution

–

Continuous Integration

Continuous Integration is the practice of

integrating early and often, so as to avoid

the pitfalls of "integration hell". The

ultimate goal is to reduce or eliminate

rework and thus reduce cost and time.

(7)

Continuous Integration

As soon as changes are made, they

are integrated with the trunk and run

through the build and test cycle

The goal is to be able to build the

code base at any time

Catch problems early, when they are

less expensive to fix

Get instant feedback on integration

problems, when the problems are

fresh in everyone’s mind

(8)

Continuous Integration Theory

Integration Build

Developer runs local

build and automated

tests

Developer checks

tested code into SCM

system

Integration build run at

frequent intervals or

upon check-in

SCM SCM System System Check-In ElectricCommander Production Servers Build Success

(9)

How do you get there?

1. Deploy an automated build and test infrastructure

• Stimulus builds: caused by changes in the SCM system

• Scheduled builds: nightly or weekly builds

2. Accelerate your builds

• Goal: One hour build and test cycle

• If necessary, choose a subset of tests to run

• Run more complete test suites during nightly or weekly scheduled builds

3. Make results visible

• The results should be visible to the entire team: emails, reports, LCD screens

4. Change your mindset

• Everyone commits code every day

• Every commit should build the mainline on an integration machine

(10)

Next Generation Continuous Integration: Preflights

Continuous Integration delivers quality and

productivity benefits

But CI can break down

The “it worked on my machine” phenomenon in a

multi-platform environment

Continuously broken builds

The answer: “preflight” builds and tests

A pilot goes through a rigorous preflight check routine prior

to takeoff

ElectricCommander preflights perform a continuous

integration build prior to checking in the code

(11)

Continuous Integration Reality

Build Failure

Developer builds/tests

on local system, checks

in code

Integration build started,

breaks (“it worked on

my machine”)

Team impacted while

check-in is backed out

or build fixed

SCM SCM System System Check-In

X

Wait

X

Wait Back Out Integration Build ElectricCommander Production Servers

(12)

Success

Preflight: Continuous Integration Evolved

Developers SCM SCM System System ElectricCommander

Test Servers Production Servers

Developers build and

test across all

targets/platforms

Ensures successful

integration build

Developers can check

in changes only upon

successful preflight

Broken builds less

likely to affect the

entire team

(13)

(14)

Case study 1:

Global mobile phone developer

Disparate processes across development organization

using different home-grown systems

Vision to:

Reduce cycle time down from hours or even days Introduce CI and enable Agile methodologies

Centralize management and administration

Enable global collaboration across sites all over the world Optimize use of hardware and software resources

Develop a scalable solution to support future growth inclusive of technology trends and sites

(15)

Case study 1:

Workflow

Setup environment

Compilation (all required targets)

Install a new clean baseline Synchronize SCM changelist

Static Code Analysis Emulator tests Hardware tests

SCA scan

Code st.

scan Build test code

Run Emulator tests

Build ROM’s

Run Hardware tests

Tear down environment

Result analysis, Report generation, Notification, Release

Coverity build Coverity analyze SCA1 build SCA2 analysis SCA scan SCA scan Line count

Schedule SCM submission IDE Preflight Manual

Run Emulator tests_{Run Emulator tests} Run Hardware tests_{Run Hardware tests} SCA scan SCA scan SCA scan SCA2 analysis_analysisSCA2

(16)

(17)

Case study 1: Status

Highly appreciated and have had a great impact

to the groups where it has been put in production

Higher quality of deliverables

Increased productivity

Typical cycle time now at ~90 minutes, target 60

Build time

~10m down from 60m

Static Analysis

~30m down from 2h 45m

Test time

~50m down from 13h 30m

(18)

Case Study 2: Broken Builds at BioWare

Game development studio at Electronic Arts with 40

developers

Manual build-and-test system

Hundreds of broken builds a month

Frustrated developers, missed schedules

Two-pronged automation and acceleration approach

Enabled preflight builds and tests, continuous integration

Reduced overall build and test cycle by 75% 90% fewer broken builds

Internal ROI case built upon two key areas

The more broken builds the higher the cost

More builds could be executed and defects found faster. Customer calculated 6X the build volume = $100 saved on a per build basis

(19)

Case Study: BioWare

Build Failures by Month

Problem:

Challenging, complicated build-test matrix (Xbox, PS3, PC)

Slow builds kept QA waiting and Developers avoiding afternoon checkins

300 broken builds per month

Solution:

ElectricAccelerator &

ElectricCommander enabled frequent build & test

Reduced build times from 2.5 hrs. to 15 min

ElectricCommander preflights provided integration-quality build and test before commit