• No results found

Scrum and Testing The end of the test role Bryan Bakker 20 maart 2012

N/A
N/A
Protected

Academic year: 2021

Share "Scrum and Testing The end of the test role Bryan Bakker 20 maart 2012"

Copied!
34
0
0

Loading.... (view fulltext now)

Full text

(1)

voordracht georganiseerd door

Discussiegroep Software Testing

(2)

Scrum and Testing ...

The end of the test role?

Bryan Bakker

ie-net March 2012

Contents

ƒ Scrum

ƒ Scrum and Testing

ƒ Scrum and Testing

ƒ More than only software

ƒ Outsourcing

ƒ Note: not everything I tell is part of scrumNote: not everything I tell is part of scrum

(3)

About Bryan Bakker

ƒ Test ArchitectTest Architect

ƒ Certifications: ISTQB, TMap, Prince2

ƒ Member of ISTQB Expert Level on Test Automation

ƒ Tutor of several test related courses

ƒ Domains: medical systems, professional security

systems, semi-industry, electron microscopy

ƒ Specialties: test automation, integration testing, design

for testability, reliability testing

© Sioux 2012 | Confidential | 3

Scrum experiences

ƒ Within Sioux almost all projects developed with ScrumWithin Sioux almost all projects developed with Scrum

ƒ Started on Scrum project in feb 2010

ƒ I am a software test expert ÆNot a Scrum Expert

(4)
(5)

Well known picture

© Sioux 2012 | Confidential | 7

Wishes and facts

ƒ Wishes:

ƒ Customer knows what he wants

ƒ Developers know how to implement it

ƒ Nothing will change along the way

ƒ Facts:

ƒ Customer discovers what he wants

ƒ Developers discover how to implement it

ƒ Most things change along the way

ƒ Most things change along the way

Æ Solution: Agile development (e.g. Scrum)

(6)

Manifesto

ƒ We have come to value:

I di id l d P d Individuals and interactions Working software Customer Processes and tools Comprehensive documentation Contract over over Customer collaboration Responding to change Contract negotiation Following a plan over over

Source: An introduction to Scrum by Mike Cohn.

© Sioux 2012 | Confidential | 9

Manifesto

ƒ We have come to value:

I di id l d P d

ƒ Less processes – more communication

ƒ More ad-hoc, problem solving

ƒ Less documented processes

ƒ Do not document everything in Quality Systems Manual

Individuals and interactions

Processes and tools over

ƒ Do not document everything in Quality Systems Manual

ƒ Tools... Too complex, too expensive

Æ Not that extreme!

(7)

Manifesto

ƒ We have come to value:

I di id l d P d

ƒ Processes still useful

ƒ Should help to solve a problem

ƒ Quality Systems Manual can still be needed

Individuals and interactions Processes and tools over y y

ƒ Standards, safety, evidence needed

ƒ Tools needed! A lot open-source is seen

ƒ Choose simple tools, that fulfil the job

Source: An introduction to Scrum by Mike Cohn.

© Sioux 2012 | Confidential | 11

Manifesto

ƒ We have come to value:

C h i

ƒ Customer wants working software not documentation

ƒ Documentation can be in the code

ƒ Spend time on software instead

over

Working software Comprehensive documentation

p

(8)

Manifesto

ƒ We have come to value:

C h i

ƒ Documentation still needed, but should be useful

ƒ Documentation can be mandatory (standards, safety,

regulatory)

ƒ Traceability evidence (don’t forget maintenance)

over

Working software Comprehensive documentation

Traceability, evidence (don t forget maintenance)

ƒ But documentation alone is not that valued...

ƒ Quality Assurance (Compliance Verification) can ensure

that correct set of documentation is delivered

© Sioux 2012 | Confidential | 13

Manifesto

ƒ We have come to value:

C t C t t

ƒ Product owner is continuously involved

ƒ Product owner can control the scope and priorities

ƒ Do not try capture everything in a contract, this is going

(9)

Manifesto

ƒ We have come to value:

R di t

ƒ Plan is nice, but plan will change

ƒ Accept the change, and embrace it

ƒ Change is an opportunity (to build the right product)

over Responding to

change Following a plan

g pp y ( g p )

© Sioux 2012 | Confidential | 15

Scrum characteristics

ƒ One of the “agile processes”

ƒ Self-organizing teamsg g

ƒ Product progresses in a series of “sprints”

ƒ Requirements are captured as items in a list of “product

backlog”

ƒ No specific engineering practices prescribed

(10)

Scrum characteristics

ƒ Features are developed in order of priority

ƒ PragmaticPragmatic

ƒ Increments result in shippable products

ƒ Daily scrum meetings to share status

ƒ Time-boxed, Q over F

ƒ Quick feedback from customer

ƒ Customer has control on projectp j

© Sioux 2012 | Confidential | 17

Scrum overview

24 hours 2-4 weeks Sprint Backlog Backlog tasks expanded by team Daily Scrum Meeting Product Backlog

As prioritized by Product Owner

Potentially Shippable Product Increment

Source: Adapted from Agile Software

Development with Scrum by Ken Schwaber and Mike Beedle.

(11)

Scrum Roles

ƒ Product Owner

ƒ Voice of the customer

ƒ Owner of product backlog (incl. prioriorities) ƒ Decides where the team should go

ƒ Not how, not how fast

ƒ Scrum Master

ƒ Focuses on scrum practices

ƒ Removes impediments -> make sure team can work

ƒ Team

ƒ Team

ƒ 5-8 persons, sit together ƒ Self organizing

ƒ Owns sprint backlog

© Sioux 2012 | Confidential | 19

Scrum Artifacts

ƒ Product backlog

ƒ High level features of project ƒ Items at top should be clear ƒ Dynamic

ƒ Sprint backlog

ƒ Contains work for sprint

ƒ Fixed for iteration (no changes in current sprint!) ƒ Quality more important than features

ƒ Visible on scrum-boardVisible on scrum board

ƒ Burndown chart

(12)

Burndown chart

burndown chart 250 100 150 200 es ti m a te d ho ur s estimated work remaining working hours

0 50 10 sept 11 sept 12 sept 13 sept 14 sept 17 sept 18 sept 19 sept 20 sept 21 sept 24 sept 25 sept 26 sept 27 sept 28 sept

days left in sprint

© Sioux 2012 | Confidential | 21

Scrum meetings

ƒ Sprint contents meeting

ƒ Which items from product backlog are added to sprint

b kl backlog

ƒ Product owner decides

ƒ Daily scrum

ƒ Stand-up meeting

ƒ Product owner not always present, but should be

(13)

Why Scrum?

ƒ Flexibility for business:

After each sprint: After each sprint:

ƒ product could be shipped

ƒ features can be added/removed on product backlog

and priorities can be changed.

ƒ Creates focus

ƒ sprint can be overseen

ƒ only spend preparation time on things that will be

ƒ only spend preparation time on things that will be

done

© Sioux 2012 | Confidential | 23

Release flexibility

ƒ Problems in old approach:

ƒ Quality level increases slowly during projectQuality level increases slowly during project

ƒ Long periods of unstable products

ƒ What would you prefer:

ƒ Getting a product soon and getting an update when it

is available, or

ƒ Wait a long time and get the final product

(14)

When to use

ƒ Suitable when:

ƒ time to market is short

ƒ time-to-market is short

ƒ desired functionality still uncertain

ƒ dynamic market situation

© Sioux 2012 | Confidential | 25

The case against

ƒ Requires trust

ƒ Too little upfront thinking:

ƒ Too little upfront thinking:

ƒ Danger of major redesign

ƒ Wrong requirements

ƒ Requires good team

(15)

Scrum and Testing

© Sioux 2012 | Confidential | 27

Test Driven Development

ƒ Not part of Scrum, based on eXtreme Programming

ƒ But used typically in Scrum projectsBut used typically in Scrum projects 1. Create a new test

2. See the new test fail

3. Write code to make test pass

4. Build, run all tests, see all tests pass, , p

(16)

Test Driven Development

ƒ Performed by software developer

ƒ Tests are continuously executedTests are continuously executed

ƒ Code is not delivered when tests fail

ƒ Testing is part of the process

ƒ Quality awareness increases

ƒ Automation is a must!

© Sioux 2012 | Confidential | 29

Test Driven Development

ƒ Add tests to regression test

ƒ Regression test is run on each buildRegression test is run on each build

ƒ Regression detected immediately

ƒ Short feedback loop

ƒ Safety net at re-designs

Æ Test Engineers should become Software Engineers

or look for another (non-agile) assignment!

(17)

Scrum and testing

ƒ TDD is important

ƒ TDD can be enough in small teamsTDD can be enough in small teams

ƒ In combination with continuous integration very strong

ƒ But test engineers can still be very useful

ƒ Tester ≠ Developer

ƒ Add a tester to each scrum team

ƒ Tester can define and execute test cases to verify

integrated functionality

© Sioux 2012 | Confidential | 31

(18)

Quadrant 1

ƒ Technology facing tests that support the team

ƒ Unit/component level

ƒ Unit/component level

ƒ TDD is used here

ƒ Focus on internal quality

ƒ Instant feedback

ƒ Build testability into code

ƒ Fully automatedFully automated

© Sioux 2012 | Confidential | 33

Quadrant 2

ƒ Business facing tests that support the team

ƒ Verify correctness of complete function

ƒ Verify correctness of complete function

ƒ Integration

ƒ If necessary use stubs and mocks

ƒ Focus on external quality

ƒ Quick feedback (during sprint)

ƒ Test engineers together with customersTest engineers together with customers

ƒ Manual and automated

(19)

Quadrant 3

ƒ Business facing tests that critique the product

ƒ Focus on validation and acceptance

ƒ Focus on validation and acceptance

ƒ Customer important

ƒ Realistic environment/use

ƒ Usability

ƒ Quick feedback (end of sprint)

ƒ Increases confidenceIncreases confidence

© Sioux 2012 | Confidential | 35

Quadrant 4

ƒ Technology facing tests that critique the product

ƒ Non functional tests

ƒ Non-functional tests

ƒ E.g. Performance, reliability, security

ƒ Stress- and load testing

ƒ Special expertise needed

ƒ Tool support needed

ƒ Probably not in early sprintsProbably not in early sprints

(20)

Role of Test Engineer

Æ Test Engineers should become Software Engineers

or look for another (non-agile) assignment!( g ) g

ƒ In fact a Test Engineer can add a lot of value

ƒ In the Scrum team

Æ Move Test Engineers from independent test teams

into scrum teams

© Sioux 2012 | Confidential | 37

More than only software

(21)

Multiple disciplines

© Sioux 2012 | Confidential | 39

Multiple disciplines - Agile

Whish Product Manager

Still magic, but smaller magic

(22)

Problems/challenges

ƒ Hardware and Software developed concurrently

ƒ Development machine not the targetDevelopment machine not the target

ƒ Used for testing (e.g. TDD) ƒ Can differ substantially

ƒ Hardware running ahead of software

ƒ Problems in hardware detected late ƒ and hopefully fixed in software

ƒ Or hardware available late in project

ƒ Or, hardware available late in project

ƒ Software cannot be tested on target hardware

© Sioux 2012 | Confidential | 41

Problems/challenges

ƒ Hard to “scrum” hardware and mechanics

ƒ But:But:

ƒ There are more deliverables than the final hardware ƒ Documents (e.g. interface specifications)

ƒ Prototypes

ƒ Alternative hardware (competitor, old-version)

ƒ Stubs and simulators (software alternative), test interfaces

ƒ Remember: we want to reduce risk as soon as possibleRemember: we want to reduce risk as soon as possible

ƒ It might be worth to spend a few euro’s on extra prototypes ƒ Although in the end, they are “thrown away”

(23)

Problems/challenges

ƒ In some environments Hardware/Mechanical

development “rather” predictablep p

ƒ Choice can be not to disturb them with a new process

ƒ Software can still benefit from Scrum

ÆWhole project can still benefit from Scrum

© Sioux 2012 | Confidential | 43

System level

ƒ Anyway...

ƒ System test activities must still be performedSystem test activities must still be performed

ƒ As soon as possible

ƒ Targets needed (might be sparse)

ƒ In sprint, but often not practical

ƒ Take system test out of sprint, and let separate team y p , p

look at this:

(24)

Independent Test Team?

System tests Integration tests System tests Integration tests Scrum System tests Integration tests Sprint n Sprint n+1 Integration tests

Development & Unit tests

Integration tests Development & Unit tests

Integration tests Development & Unit tests

Integration tests Development & Unit tests

Release Scrum Team: Team: Release Sprint n+2 Integration tests Development & Unit tests

Release

Integration tests Development & Unit tests

Sprint n Sprint n+1 System tests p p Release Independent Test Team: Internal

Release InternalRelease

Sprint n+2 p

Internal Release

© Sioux 2012 | Confidential | 45

Multiple Scrum teams

(25)

Multiple Scrum teams

Potentially

Sprint

Contents Sprint Team A

Potentially Shippable

Product

Sprint

Contents Sprint Team B

Potentially Shippable

Product

Sprint

Contents Sprint Team C

Potentially Shippable

Product

© Sioux 2012 | Confidential | 47

Multiple Scrum teams

Sprint

C t t Sprint Team A CSprintt t Sprint Team A Contents Sprint Team A

Potentially Shippable Product Sprint

Contents Sprint Team B

Sprint

Contents Sprint Team C

Contents Sprint Team A

Potentially Shippable Product Sprint

Contents Sprint Team B

Sprint

Contents Sprint Team C

(26)

Role of Test Engineer

Æ Move Test Engineers from independent test teams

into scrum teams

ƒ Independent test engineers still needed

ƒ Probably less, so some can move to scrum teams

© Sioux 2012 | Confidential | 49

What about regulations?

ƒ Satisfying standards, regulations each sprint?

ƒ Probably notProbably not...

ƒ Mandatory at the end of development (End Game)

ƒ Think of interim releases

ƒ Overhead can be large (e.g. FDA approval)

ƒ But benefit can be highg

ƒ Customer feedback

ƒ Problems with standards, safety, etc detected early

Æearly risk reduction

(27)

Evidence… traceability?

ƒ When formal process is needed to release a product

ƒ Safetyy

ƒ Industry standards

ƒ Regulatory approvals (e.g. FDA)

ƒ Overall quality characteristics

Best practise:

ƒ Can be done outside the Scrum teams

ƒ Can be done outside the Scrum teams

ƒ By independent teams

ƒ Test, regulatory, quality, (hardware) standards, ...

ƒ Link to the Scrum teams, but still independent

© Sioux 2012 | Confidential | 51

Scrum and CMMI

ƒ Scrum focuses on people and interaction.

ƒ CMMI focuses on processes

ƒ CMMI focuses on processes.

ƒ Scrum focuses on working software.

ƒ CMMI focuses on documentation.

ƒ CMMI and Scrum can be combined, but their natural

(28)

Outsourcing

© Sioux 2012 | Confidential | 53

Common issues with

outsourcing

ƒ Difficult communication

ƒ Cultural differencesCultural differences

ƒ Limited domain and system knowledge

ƒ Requirements must be clear and very detailed

ƒ Time-zone difference

ƒ Not under control

ƒ Focus

(29)

Scrum and outsourcing?

Some bullets from other sheets: ƒ Self-organizing teams

ƒ Daily scrum meetings to share status

ƒ Requirements are captured as items in a list of “product backlog”

ƒ Quick feedback from customer

ƒ Self-steering teams

The whole team takes a shared responsibility on achieving the project goal (accountability and authority).

ƒ desired functionality still uncertain

Æ Don’t combine Scrum and outsourcing

© Sioux 2012 | Confidential | 55

Sioux and outsourcing

ƒ Moscow

ƒ Availability of skilled engineersAvailability of skilled engineers

ƒ European culture

ƒ English speaking

ƒ Minimal time zone difference (2 hours, but...)

ƒ Frequent travelling possible

ƒ Of course: lower hour rate

ƒ 2 customers are using this service

(30)

Wish, need Release

V-Model

Integration Test Architecture System

Requirements System Test

User

Requirements Acceptance Test

FrontOffice Eindhoven Implementation Component Test Design Integration Test Architecture BackOffice Moscow © Sioux 2012 | Confidential | 57

FO – BO responsibilities

ƒ FO

ƒ Requirements elicitation / discussions with customerq

ƒ Architecture (high level)

ƒ Project management

ƒ BO

ƒ Architecture (low level)

ƒ Design

ƒ Implementation & Test (simulation)

ƒ Implementation & Test (simulation)

ƒ FO

ƒ Integration and system test (targets)

ƒ Reporting

(31)

Outsourcing and Scrum

ƒ Sprint backlog defined by FO+BO and customer

ƒ Sprints of 3 weeksSprints of 3 weeks

ƒ Daily scrums

ƒ Demo’s given by FO to customer

ƒ Retrospectives with the team

ƒ FO present at customers site (regularly)

ƒ Some extra tools needed:

ƒ Skype, WebEx, electronic scrum-board

© Sioux 2012 | Confidential | 59

(32)

Advantages

ƒ Daily scrums overcome multiple problems:

ƒ CommunicationsCommunications

ƒ Focus

ƒ Control

ƒ FO-BO construction overcomes:

ƒ Domain and system knowledgey g

ƒ Requirements

Æ Don’t combine Scrum and outsourcing

© Sioux 2012 | Confidential | 61

Final thoughts

ƒ Scrum is not the solution for all problems

ƒ You still need good peopleYou still need good people

ƒ Scrum is not new

ƒ Focus on team work and quick feedback

ƒ (Independent) Testing still necessary

ƒ Scrum possible in (multi-disc.) product development

ƒ Scrum helps when outsourcing is appliedp g pp

(33)
(34)

                             

Consulteer onze website

) http://www.ie-net.be

References

Related documents

Product Backlog & Team Formation Sprint Planning 2 Parts: Selection and Decomp Daily Scrum Sprint 2-4 Weeks Team Retrospective..

3 roles • Product owner • Scrum master • Team 3 artifacts • Product backlog • Sprint backlog • Sprint burndown 3 ceremonies • Sprint planning • Daily scrum • Sprint

Sprint Planning Meeting Daily Scrum Meeting Sprint Retrospective 7 8 9 10 11 12 5 6 13 SCRUM Product Backlog.. • Scrum allows teams of people to develop complex

The training involves the main Scrum practices were introduced to the team including sprint zero, product backlog, sprint backlog, sprint planning meetings, daily scrum

ScrumMaster Shippable Product Sprint Planning Scrum Team Product Backlog Daily Scrum Product Owner Sprint Backlog Sprint Review..

SCRUM Practices Product Backlog Sprint Backlog Release Backlog Sprint Retrospective New Functionality Sprint Plan Scrum Meeting 24 hours Begin Sprint End Sprint 30 days. Other

Mountain Goat Software, LLC • Product owner • ScrumMaster • Team Roles Scrum framework • Product backlog • Sprint backlog • Burndown charts Artifacts • Sprint planning

„ Product Owner identifies Product Backlog items „ (Development Team estimates Product Backlog) „ (Product Owner prioritizes Product Backlog).. Scrum