The Definition of Metrics for Continuous Integration in SCRUM. How Continuous Is Our Continuous Integration?

25  Download (0)

Full text

(1)

The Definition of Metrics for Continuous

Integration in SCRUM

––––––

How Continuous Is Our Continuous

Integration?

Christian Facchi

University of Applied Sciences

I l t dt

Ingolstadt

Jochen Wessel Jochen Wessel

Nokia Siemens Networks SMEF 2010

(2)

Outline

SCRUM and Continuous Integration

1

Definition of Metrics for Continuous Integration

2

Implementation

3

4 Case Study

Conclusion & Future Work

(3)

A il M if t Agile Manifesto:

ƒ Individuals and interactions over processes and tools

ƒ Individuals and interactions over processes and tools

ƒ Working software over comprehensive documentation

ƒ Customer collaborationCustomer collaboration over contract negotiationover contract negotiation

ƒ Respond to change over following a plan

Î small feedback cycles

Î focus on product /deliveries

H t i b d?

(4)
(5)

S b i i i l f SCRUM Some basic principles of SCRUM:

ƒ continuous feedback loop

ƒ continuous feedback loop

ƒ every backlog item should produce deliverable code Supported by continuous integration:

ƒ complete build and test after every modification (checkin)

ƒ if build or test fails problems are solved

Î early detection of integration problems (interfaces)

Î bi b t th d f th j t/i t

(6)

R i t f C ti I t ti

Requirements for Continuous Integration:

ƒ automated build and test

ƒ automated build and test

ƒ automated configuration management Available toolsupport (visualisation):

ƒ Cruise control

ƒ Hudson

(7)

Software Development

Project Landscape

Software Development

UMTS/LTE Telecommunication System

WCDMA e cifi c a ti o n U P S W LTE GSM WCDMA e cifi c a ti o n U P S W LTE GSM UMTS / LTE o n W WCDMA SW RA T s p e Applic a CP and U LTE SW GSM SW o n W WCDMA SW RA T s p e Applic a CP and U LTE SW GSM SW n BTS Co mm o A pplicat io n S BTS Site M RFSW Common Application SW Co mm on SW TRSW n BTS Co mm o A pplicat io n S BTS Site M RFSW Common Application SW Co mm on SW TRSW B T S C o mmo n Pl at fo rm SW Mgr FR/AAS BTS

Common Platform Services SW

B T S C o mmo n Pl at fo rm SW Mgr FR/AAS BTS

Common Platform Services SW

Flexi BTS System Module HW

FR/AAS Module HW

PC/

Workstation Flexi BTS System Module HW

FR/AAS Module HW

PC/ Workstation

(8)

Wh t i f C ti I t ti Why metrics for Continuous Integration

ƒ no objective input from SCRUM teams only feelings

ƒ no objective input from SCRUM teams only feelings

ƒ prioritisation of impediments among different SCRUMprioritisation of impediments among different SCRUM teams

Î these metrics can be used not only in SCRUM

Î i t t t d b ild

(9)

SCRUM and Continuous Integration 1

Definition of Metrics for Continuous Integration 2

Implementation

3

4 Case Study

Conclusion & Future Work

(10)

P l

f C

ti

I t

ti

Pulse of Continuous Integration

CI frequency: = number of successful integrations per day CI frequency: = number of successful integrations per day ƒ heartbeat of the SCRUM team

ƒ heartbeat of the SCRUM team

ƒ target is CI frequency ≥ 1* |developers|g q y | p |

ƒ only successful integrations are measured (correct delivery of a product); not erroneous checkin (error correction phase)

(11)

Q

lit

f C

ti

I t

ti

Quality of Continuous Integration

ƒ #CI{OK ERR}{OK,ERR} are defined as number of correct or erroneous checkins

ƒ ErrorRatio has been determined for each phase (build,

itt t t t t )

unittest, systemtest, …)

ƒ indicator for:

ƒ indicator for:

ƒ process problems (before checkin a successful test is required)

(12)

D

ti

f C

ti

I t

ti

Duration of Continuous Integration

ƒ defined as maximal value of an automated tool completion

ƒ defined as maximal value of an automated tool completion time per day for a special development phase of a

component

ƒ if e.g. the build time is to long the number of integration problems will grow, due to parallel checkins

ƒ as target limit 10 minutes

(13)

A il billit

f C

ti

I t

ti

Availabillity of Continuos Integration

uptime := time where system in <phase> is up and running

uptime<phase> := time where system in <phase> is up and running

downtime := time where system in <phase> is not running

downtime<phase> := time where system in <phase> is not running

ƒ availabillity should be very high (basic requirement for

d l t t k

development tasks

(14)

SCRUM and Continuous Integration 1

Definition of Metrics for Continuous Integration

2

Implementation 3

4 Case Study

Conclusion & Future Work

(15)

I

l

t ti

Implementation:

ƒ fully automated test and build environment (UNIX scripts)

ƒ fully automated test and build environment (UNIX scripts)

ƒ extension of scripts to write metrics information

ƒ extension of scripts to write metrics information

(component name, phase, state, timestamp, …) in logfiles

METRICS < t>< h >< t t ><ti > METRICS:<component><phase><state><time>

ƒ state = {started| ok| failed}

ƒ state = {started| ok| failed}

ƒ PERL scripts to analyze logfilesPERL scripts to analyze logfiles

(16)

SCRUM and Continuous Integration 1

Definition of Metrics for Continuous Integration

2

Implementation

3

4 Case Study

Conclusion & Future Work

(17)

Case study:

ƒ 2 SCRUM teams

d l t it

ƒ one development site

ƒ one team in bugfix phase and one team in development phase Observations:

ƒ low pulse of integration: less than 1 per team member and day; reason: already known task split (developer, tester)

ƒ constant pulse of integration; no rush hour at the end of a sprint;

Î ok

ƒ slow development cycles:

ƒ slow development cycles:

ƒ long maximal build time (up to 1.5 hours)

ƒ long maximal test time (up to 2 hours)long maximal test time (up to 2 hours)

(18)

Actions:

ƒ buildprocess has been optimized

Î immediately reduced to 50% further improvement

Î immediately reduced to 50%, further improvement scheduled

ƒ remove tasksplitt between team members (tester, developer); areas of competence destabilized

Î long term issue (education) Advantages for SCRUM teams:

ƒ improved buildtime leads to reduce cycle time and to less integration problems (not reported)

(19)

Results of the case study:

ƒ proposed metrics are very helpful for SCRUM teams. Despite the fact that metrics are suspected in general Despite the fact that metrics are suspected in general

ƒ analyzing metrics needs time and deep insight (wrong y g p g ( g conclusions can be drawn very fast)

ƒ objective results to achieve improvements (measureable) less is more (concentrate on some al able metrics)

ƒ less is more (concentrate on some valuable metrics)

ƒ better visualisation should help SCRUM teams (see burn

ƒ better visualisation should help SCRUM teams (see burn down chart)

(20)

SCRUM and Continuous Integration 1

Definition of Metrics for Continuous Integration

2

Implementation

3

4 Case Study

Conclusion & Future Work 5

(21)

Conclusion

Summary

Summary

ƒ metrics in an agile environment are useful

ƒ careful selection of metrics

ƒ careful interpretation of results

ƒ to much metrics leads to control by management (contradiction to SCRUM)

( )

Î metrics should help SCRUM teams, so the teams should define the metrics

ƒ best practice: Continuous Integration can be used in every SW development process: Try itp p y

(22)

Future Work

Wh t

i

t b d

?

What remains to be done?

li ti t th ( l d i j t th il

application to other areas (plan driven projects; other agile

methods)

better visualization (Integration in Continuous Integration

tooling); feedback to SCRUM teams

(23)

Thank you for your attention

Questions?

Q

University of Applied Sciences Ingolstadt Hochschule Ingolst adt

Prof. Dr. Christian Facchi

SW Engineering and Distributed Systems Esplanade 10 85049 Ingolstadt Phone: +49-841-9348-365 F +49 841 9348220 Fax: +49-841-9348220 E-mail: christian.facchi@haw-ingolstadt.de www: www.haw-ingolstadt.de/

(24)

SCRUM t LTE C Pl SW i N ki Si N t k SCRUM at LTE C-Plane SW in Nokia Siemens Networks

(environment):

ƒ development of a part of eNodeB telecommunication SW (LTE C-Plane)

ƒ ~90 software developers

ƒ 2 development sites 8 SCRUM t

ƒ 8 SCRUM teams

ƒ embedded in bigger frame program (using waterfall model)

(25)

S i l t f SCRUM t LTE C Pl SW i N ki Special setup of SCRUM at LTE C-Plane SW in Nokia

Siemens Networks:

ƒ Feature Expert Team:

ƒ interface to traditional (waterfall) environment (frame process)

ƒ continuity of development

ƒ introduction of features to SCRUM teams

ƒ introduction of features to SCRUM teams

ƒ support of Product Owner to build up backlog

ƒ Feature Expert Team has been divided furthermore:

ƒ Subject Matter Expertsj p

ƒ one central Product Owner and one proxy Product owner per site

kl SCRUM f SCRUM

Figure

Updating...

References

Related subjects :