Continuous Integration

(1)

Continuous Integration

CODING & DEVELOPMENT | BORIS GORDON | FEBRUARY 7 2013

(2)

Introduction

About me

•

boztek on d.o. (http://drupal.org/user/134410)

•

@boztek

•

[email protected]

2

(3)

Introduction

About you

3

(4)

Introduction

About you

3

•

Probably a developer

(5)

Introduction

About you

3

•

Maybe don’t know what CI is at all

(6)

Introduction

About you

3

•

Probably have some idea but don’t know where to start

(7)

Introduction

About you

3

•

Wondering how/if it applies to Drupal projects

(8)

Introduction

About you

3

•

Wondering how/if it applies to Drupal projects

•

Have some “scaling” concerns

(9)

Introduction

What we will cover

4

(10)

Introduction

What we will cover

4

•

Motivation

(11)

Introduction

What we will cover

4

•

Motivation

•

Continuous integration basics

(12)

Introduction

What we will cover

4

•

Motivation

•

Getting started

(13)

Introduction

What we will cover

4

•

Motivation

•

Getting started

•

Best practices

(14)

Introduction

What we will cover

4

•

Motivation

•

Getting started

•

Best practices

•

Building, testing and deploying

(15)

Introduction

What we will cover

4

•

Motivation

•

Getting started

•

Best practices

•

Building, testing and deploying

•

Continuous integration servers

(16)

Introduction

What we will

not

cover

5

(17)

Introduction

What we will

not

cover

5

•

Which software you need to install to integrate for you

(18)

Introduction

What we will

not

cover

5

•

The perfect one size fits all project template

(19)

Introduction

What we will

not

cover

5

•

How to install Jenkins

(20)

Introduction

What we will

not

cover

5

•

How to write Drupal tests

(21)

Introduction

What we will

not

cover

5

•

How to write Drupal tests

•

I won’t really try to convince you that should be doing this

(22)

What’s the problem?

We are talking about construction

6

(23)

What’s the problem?

We are talking about construction

6

(24)

What’s the problem?

We are talking about construction

1. Pre-build (requirements, specs, etc.)

6

(25)

What’s the problem?

We are talking about construction

1. Pre-build (requirements, specs, etc.)

2. Design and develop and test locally

6

(26)

What’s the problem?

We are talking about construction

1. Pre-build (requirements, specs, etc.)

2. Design and develop and test locally

3. Deploy release to test server

6

(27)

What’s the problem?

We are talking about construction

1. Pre-build (requirements, specs, etc.)

2. Design and develop and test locally

3. Deploy release to test server

4. Testers review

6

(28)

What’s the problem?

We are talking about construction

1. Pre-build (requirements, specs, etc.)

2. Design and develop and test locally

3. Deploy release to test server

4. Testers review

5. Fix bugs

6

(29)

What’s the problem?

We are talking about construction

1. Pre-build (requirements, specs, etc.)

2. Design and develop and test locally

3. Deploy release to test server

4. Testers review

5. Fix bugs

6. Testers review

6

(30)

What’s the problem?

We are talking about construction

1. Pre-build (requirements, specs, etc.)

2. Design and develop and test locally

3. Deploy release to test server

4. Testers review

5. Fix bugs

6. Testers review

7. Fix bugs

6 Saturday, 9 February 13

(31)

What’s the problem?

We are talking about construction

1. Pre-build (requirements, specs, etc.)

2. Design and develop and test locally

3. Deploy release to test server

4. Testers review

5. Fix bugs

6. Testers review

7. Fix bugs

8. Deliver

(32)

What’s the problem?

We are talking about construction

1. Pre-build (requirements, specs, etc.)

2. Design and develop and test locally

3. Deploy release to test server

4. Testers review

5. Fix bugs

6. Testers review

7. Fix bugs

8. Deliver

(33)

What’s the problem?

We are talking about construction

1. Pre-build (requirements, specs, etc.)

2. Design and develop and test locally

3. Deploy release to test server

4. Testers review

5. Fix bugs

6. Testers review

7. Fix bugs

8. Deliver

(34)

Software Engineering 101

Not releasing defects is much cheaper than

fixing them later.

8

(35)

How can we fix?

Turns out it’s cheaper to fail faster and more

often.

9

(36)

How can we fix?

Problems with divide and conquer

10

(37)

How can we fix?

Integration is waste right?

11

(38)

How can we fix?

Integration is waste right?

11

(39)

“If it feels good, do it”?

12

(40)

“If it feels good, do it”?

Maybe if it’s painful, you aren’t doing it

enough

13

(41)

“If it feels good, do it”?

Maybe if it’s painful, you aren’t doing it

enough

13

(42)

“If it feels good, do it”?

Maybe if it’s painful, you aren’t doing it

enough

•

Integrating small changes reveals problems earlier

13

(43)

“If it feels good, do it”?

Maybe if it’s painful, you aren’t doing it

enough

•

Integrating regularly produces feedback quicker than integrating sporadically

13

(44)

“If it feels good, do it”?

Maybe if it’s painful, you aren’t doing it

enough

•

13

(45)

“If it feels good, do it”?

Maybe if it’s painful, you aren’t doing it

enough

•

Maybe we should just integrate all the time?

13

(46)

Continuous integration

Is that even possible?

14

(47)

Continuous integration

Is that even possible?

14

(48)

Continuous integration

Is that even possible?

•

Yes

14

(49)

Continuous integration

Is that even possible?

•

Yes

•

... sort of

14

(50)

Continuous integration

Is that even possible?

•

Yes

•

... sort of

•

... with discipline

14

(51)

Continuous integration

Is that even possible?

•

Yes

•

... sort of

•

... with discipline

•

... and sufficient tools

14

(52)

Continuous integration

How continuous is continuously?

“Continuously is more often than

you think”

- Jez Humble

15

(53)

CI is a practice

It’s not really about tools

It is about getting past our

adolescent surety that “I know better

than everyone else and all I need is

to be left alone to be the greatest.”

- Kent Beck

16

(54)

The CI mindset

•

Without CI : consider your application’s default state broken (aka “under development”)

•

With CI : the starting point for any development work is a stable, robust, tested application

•

Mistakes are made early and quickly and addressed

when it’s cheapest to fix

Goals and principles

17

(55)

CI is a practice

18

In other words

•

“apt-get install jenkins” !== CI

•

Installing and maintaining a CI server may be an IT/Ops job

•

Doing CI is something the developers practice

(56)

Defining some terms

Getting started

19

(57)

Defining some terms

•

build : some version-able, deployable, stand alone artefacts that can be tracked across different

environments

Getting started

19

(58)

Defining some terms

•

environments

•

test : examine build artefacts and has own artefacts

Getting started

19

(59)

Defining some terms

•

environments

•

test : examine build artefacts and has own artefacts

•

deploy : moving a build around

Getting started

19

(60)

Start with the essentials

•

check into mainline regularly

•

test everything (that can break)

•

building and testing should be fast

Implementing CI

20

(61)

Regular “mainline checkins”

Implementing CI

21

(62)

Comprehensive automated tests

Confidence in a build is dependent on the breadth and quality of tests.

•

Testing needs to be as frictionless as possible

•

Tests need to run fast

•

Drupal simpletest is pretty slow

•

But there are other types of tests and you don’t have to run all of them all the time

Implementing CI

22

(63)

Comprehensive automated tests

Some types of tests and recommended tools. 1. Unit tests - PHPUnit - DrupalUnittestCase 2. Functional tests - Simpletest 3. Smoke tests

- Shell scripts, simpletest

4. Regression tests (use one of the above) 5. Automated acceptance tests

- Behat with mink or selenium

Implementing CI

23

(64)

Comprehensive automated tests

That’s a lot of tests!

Different types of tests can be run at different stages of a CI pipeline

Implementing CI

24

(65)

Productive development environments

Consistent development environments are important.

•

Writing tests needs to be as frictionless as possible too

•

Tests need to run fast locally

•

Ideally all tests need to be runnable locally

Implementing CI

25

(66)

Productive development environments

Configuration and data should be versioned

•

Build scripts

•

Test data

•

Deploy configurations

•

Dependency management

Implementing CI

(67)

Productive development environments

Building and testing should be

fast

Implementing CI

27

(68)

Productive development environments

Why bother?

Like TDD it can reveal bad smells you might otherwise ignore

Implementing CI

28

(69)

Some best practices

Doing Continuous Integration

29

(70)

Some best practices

•

Don’t check in on a broken build

Doing Continuous Integration

29

(71)

Some best practices

•

Run tests (locally) before merging into the main line

Doing Continuous Integration

29

(72)

Some best practices

•

Keep the main line build stable

Doing Continuous Integration

29

(73)

Some best practices

•

Know how to manage instability

Doing Continuous Integration

29

(74)

Some best practices

•

Take responsibility

Doing Continuous Integration

29

(75)

Some best practices

•

Take responsibility

•

Don’t forget about the architecture

Doing Continuous Integration

29

(76)

Some best practices

•

Take responsibility

•

Beware slow tests

Doing Continuous Integration

29

(77)

Some best practices

•

Take responsibility

•

Beware slow tests

•

Code style matters

Doing Continuous Integration

29

(78)

Build

An automated build.

This is the best place to start.

Managing a Drupal Pipeline

30

(79)

Build

Managing a Drupal Pipeline

31

(80)

Build

. !"" README.md !"" build !"" build.properties !"" build.properties.dist !"" build.xml !"" db !"" src !"" test #"" tools

Managing a Drupal Pipeline

32

(81)

Build - Demo

$> phing

Managing a Drupal Pipeline

33

(82)

Test

Unit tests and integration tests are really part of the build. Static analysis is part of the build.

Managing a Drupal Pipeline

34

(83)

Test

So we can run tests with Phing.

Managing a Drupal Pipeline

35

(84)

Deploy

•

Capistrano

•

Phing

•

Environment differences

Managing a Drupal Pipeline

36

(85)

Automate all the builds!

Once you have automated builds, automated tests and automated deployments it’s time to wire them up.

•

Jenkins

•

Travis CI

•

ThoughtWorks Go

Continuous Integration Servers

37

(86)

Tie it all together with robots

A CI server is the robot that sits between your version control system and your build system.

Robots can do heavier lifting

Continuous Integration Servers

38

(87)

Jenkins

Jenkins is a popular CI server that provides the hub

between your version control and build / test / deploy tools.

•

VCS integrations

•

Reporting

•

Triggering

•

Configurable UI / dashboard

Continuous Integration Servers

39

(88)

Travis CI

Continuous Integration Servers

40

(89)

References and further reading

Continuous Delivery by Jez Humble

http://www.amazon.com/Continuous-Delivery-Deployment-Automation-Addison-Wesley/dp/0321601912

Continuous Integration by Duvall, Matyas and Glover

http://www.amazon.com/Continuous-Integration-Improving-Software-Reducing/dp/ 0321336380/ref=sr_1_1?

s=books&ie=UTF8&qid=1360206726&sr=1-1&keywords=continuous+integration

Drupal phing build.xml

http://reload.github.com/phing-drupal-template/

Continuous Integration Servers

41

(90)

References and further reading

http://github.com/boztek/drupal-project-template

Continuous Integration Servers

42

(91)

Integration complete

Questions

•

boztek on d.o. (http://drupal.org/user/134410)

•

@boztek

•

[email protected]

43