How To Write Unit Tests In A Continuous Integration

(1)

Continuous

Integration

(2)

1. “It works

on my

(3)

(4)

(5)

Risk 1

Lack of Deployable

Software

(6)

(7)

(8)

Risk 2

(9)

(10)

(11)

Risk 3

(12)

(13)

(14)

Risk 4

(15)

All these risks are related to

(16)

Old School Development

Version Control System Lead Developer Developer Developer Commit Code Builds sources Write Code Write Code Write Code Get latest sources

(17)

Manual

Effort

Slow

Feedback

Debugging

(18)

(19)

(20)

Build software

(21)

Development

practice

where

team

members integrate their work

daily

verified by

automated

builds

using different

testing

types

(22)

Development

practice

where

team

members integrate their work

daily

verified by

automated

builds

using different

testing

types

(23)

3 building blocks

of continuous

(24)

(25)

</CODE>

(26)

(27)

</CODE>

(28)

(29)

</CODE>

(30)

7 Principles of

Continuous

(31)

(32)

Check in at least once

a day

Small Steps are beter

than one big leap

Check in after each

task

Avoid the 5PM check

in mess

(33)

we can check in as often

as we want…

(34)

Continuous Integration

without testing

=

(35)

1. Commit code frequently

(36)

(37)

(38)

(39)

</CODE>

(40)

</CODE>

(41)

</CODE>

(42)

1. Commit code frequently

2. Write automated unit tests

(43)

</CODE>

(44)

</CODE>

(45)

</CODE>

(46)

1. Commit code frequently

2. Write automated unit tests

3. Run private builds

(47)

</CODE>

(48)

</CODE>

(49)

</CODE>

(50)

1. Commit code frequently

2. Write automated unit tests

3. Run private builds

4. Don’t commit broken code

(51)

You Break It

(The Build)

You Buy It

(Round of Beer)

You Fix It

(Quickly)

(52)

1. Commit code frequently

2. Write automated unit tests

3. Run private builds

4. Don’t commit broken code

5. Fix broken build immediately

(53)

(54)

1. Commit code frequently

2. Write automated unit tests

3. Run private builds

4. Don’t commit broken code

5. Fix broken build immediately

6. Avoid getting broken code

(55)

</CODE>

(56)

</CODE>

(57)

Development

practice

where

team

members integrate their work

daily

verified by

automated

builds

using different

testing

types

(58)

(59)

one team

(60)

one team

Developers

Testers

(61)

Development

practice

where

team

members integrate their work

daily

verified by

automated

builds

using different

testing

types

(62)

(63)

UI

Service/API

Unit

Test Execution Time

(64)

(65)

(66)

(67)

(68)

(69)

(70)

(71)

1 Unit – 99,5% reliable

1 system = 50 Units

(72)

(73)

(74)

(75)

(76)

About Unit Testing

■

Verify behavior of small elements in a software system,

most often a single class

■

Occasionally more than 1 class in case of tightly coupled

classes

■

No connections to DB, filesystem,…

■

Code Coverage per test is generally limited

■

Should be very fast

■

Crucial Component of CI, unit tests run in every build

(77)

Where in CI?

VCS CI Server Developer Feedback Mechanism Commit Code + Unit Tests Polls for changes Builds sources Unit Tests Build Server

(78)

Build the code RIGHT

(79)

About Unit Integration/Acceptance Testing

■

Also a white box test

■

May cross architectural boundaries

■

More code coverage per test

■

Longer running duration than Unit Tests

(80)

Where in CI?

VCS CI Server Developer Feedback Mechanism Commit Code + Unit Tests Polls for

changes Builds sources

Unit Tests Send Feedback Unit Integration Tests Commit Build (Lightweight) Secondary Build (Heavyweight) If Commit Build Succesful

(81)

JUnit

NUnit

MS Tests

TestNG

DBUnit

Some Namedropping

(82)

Unit

Performance

Testing

(83)

(84)

(85)

(86)

(87)

Where in CI?

Unit Tests Send Feedback Unit Integration Tests Commit Build (Lightweight) Secondary Build (Heavyweight) If Commit Build Succesful Unit Performance Tests

(88)

JUnitPerf

JUnitBench

ContiPerf

NUnitPerf

(89)

API/Service

Testing

(90)

(91)

(92)

Run Test Case

(93)

Fixture

(94)

The actual Code

(95)

Run Test Case

(96)

Fix Code

(97)

Run Test Case

(98)

Refactor Code

(99)

(100)

(101)

(102)

FitNesse

Cucumber

Concordion

SOAPUI

JMeter

HP ServiceTest

Some Namedropping

(103)

Where in CI?

Unit Tests Send Feedback Unit Integration Tests Commit Build (Lightweight) Secondary Build (Heavyweight) Unit Performance Tests Service/API Tests

(104)

(105)

We’re not done yet!

• Selenium

• HP Unified Functional Testing - FASTBoX • MS Coded UI Testing

GUI Automated Testing

• JMeter

• HP LoadRunner • MS Visual Studio • RadView WebLoad

E2E Performance Testing

• HP WebInspect • HP QAInspect

(106)

Where in CI?

Unit Tests Send Feedback Unit Integration Tests Commit Build (Lightweight) Secondary Build (Heavyweight) Unit Performance Tests Nightly Build Service/API Tests GUI Automated E2E Performance Dynamic Security

(107)

Mapping Tests

on Build Types

(108)

Builds Strategy

• Unit Tests

Lightweight Commit Build

• Unit Integration • Unit Performance • API/Service Test

• Automated UI Smoke Tests

Secondary Build (Heavyweight)

• Automated UI Tests • Static Testing

• Static Application Security Testing • Dynamic Application Security Testing

Nightly Build

• Automated UI Tests • Manual Tests

(109)

Manual Testing

& CI

(110)

(111)

(112)

(113)

(114)

(115)

one team

Developers

Testers

(116)

one team

Developers

(117)

(118)

(119)

(120)

(121)

(122)

Reduce project

(123)

Reduce

repetitive

(124)

Generate

deployable

(125)

Remember our

project risks?

(126)

Lack of Deployable Software

Risk 1

(127)

(128)

(129)

Lack of Project Visibility

Lack of Deployable Software

Risk 1

(130)

Heatmap with LoC (Size), and Code Coverage (Color)

(131)

(132)

Lack of Project Visibility

Lack of Deployable Software

Low Structural Quality

Risk 1

Risk 2

(133)

(134)

Level of Code Duplication

(135)

Lack of Project Visibility

Lack of Deployable Software

Low Structural Quality

Late Discovery of Defects

Risk 1

Risk 2

Risk 3

(136)

Builds Strategy

• Unit Tests

Lightweight Commit Build

• Unit Integration • Unit Performance • API/Service Test • UI Smoke Tests

Secondary Build (Heavyweight)

• E2E Performance Testing • Static Testing

• Static Application Security Testing • Dynamic Security Testing

Non-Functional Build (Nightly/Weekly)

• Manual Testing • UI Tests

(137)

Lack of Project Visibility

Lack of Deployable Software

Low Structural Quality

Late Discovery of Defects

Risk 1

Risk 2

Risk 3

(138)

(139)