voordracht georganiseerd door
Discussiegroep Software Testing
Scrum and Testing ...
The end of the test role?
Bryan Bakker
ie-net March 2012
Contents
Scrum
Scrum and Testing
Scrum and Testing
More than only software
Outsourcing
Note: not everything I tell is part of scrumNote: not everything I tell is part of scrum
About Bryan Bakker
Test ArchitectTest Architect
Certifications: ISTQB, TMap, Prince2
Member of ISTQB Expert Level on Test Automation
Tutor of several test related courses
Domains: medical systems, professional security
systems, semi-industry, electron microscopy
Specialties: test automation, integration testing, design
for testability, reliability testing
© Sioux 2012 | Confidential | 3
Scrum experiences
Within Sioux almost all projects developed with ScrumWithin Sioux almost all projects developed with Scrum
Started on Scrum project in feb 2010
I am a software test expert ÆNot a Scrum Expert
Well known picture
© Sioux 2012 | Confidential | 7
Wishes and facts
Wishes:
Customer knows what he wants
Developers know how to implement it
Nothing will change along the way
Facts:
Customer discovers what he wants
Developers discover how to implement it
Most things change along the way
Most things change along the way
Æ Solution: Agile development (e.g. Scrum)
Manifesto
We have come to value:
I di id l d P d Individuals and interactions Working software Customer Processes and tools Comprehensive documentation Contract over over Customer collaboration Responding to change Contract negotiation Following a plan over over
Source: An introduction to Scrum by Mike Cohn.
© Sioux 2012 | Confidential | 9
Manifesto
We have come to value:
I di id l d P d
Less processes – more communication
More ad-hoc, problem solving
Less documented processes
Do not document everything in Quality Systems Manual
Individuals and interactions
Processes and tools over
Do not document everything in Quality Systems Manual
Tools... Too complex, too expensive
Æ Not that extreme!
Manifesto
We have come to value:
I di id l d P d
Processes still useful
Should help to solve a problem
Quality Systems Manual can still be needed
Individuals and interactions Processes and tools over y y
Standards, safety, evidence needed
Tools needed! A lot open-source is seen
Choose simple tools, that fulfil the job
Source: An introduction to Scrum by Mike Cohn.
© Sioux 2012 | Confidential | 11
Manifesto
We have come to value:
C h i
Customer wants working software not documentation
Documentation can be in the code
Spend time on software instead
over
Working software Comprehensive documentation
p
Manifesto
We have come to value:
C h i
Documentation still needed, but should be useful
Documentation can be mandatory (standards, safety,
regulatory)
Traceability evidence (don’t forget maintenance)
over
Working software Comprehensive documentation
Traceability, evidence (don t forget maintenance)
But documentation alone is not that valued...
Quality Assurance (Compliance Verification) can ensure
that correct set of documentation is delivered
© Sioux 2012 | Confidential | 13
Manifesto
We have come to value:
C t C t t
Product owner is continuously involved
Product owner can control the scope and priorities
Do not try capture everything in a contract, this is going
Manifesto
We have come to value:
R di t
Plan is nice, but plan will change
Accept the change, and embrace it
Change is an opportunity (to build the right product)
over Responding to
change Following a plan
g pp y ( g p )
© Sioux 2012 | Confidential | 15
Scrum characteristics
One of the “agile processes”
Self-organizing teamsg g
Product progresses in a series of “sprints”
Requirements are captured as items in a list of “product
backlog”
No specific engineering practices prescribed
Scrum characteristics
Features are developed in order of priority
PragmaticPragmatic
Increments result in shippable products
Daily scrum meetings to share status
Time-boxed, Q over F
Quick feedback from customer
Customer has control on projectp j
© Sioux 2012 | Confidential | 17
Scrum overview
24 hours 2-4 weeks Sprint Backlog Backlog tasks expanded by team Daily Scrum Meeting Product BacklogAs prioritized by Product Owner
Potentially Shippable Product Increment
Source: Adapted from Agile Software
Development with Scrum by Ken Schwaber and Mike Beedle.
Scrum Roles
Product Owner
Voice of the customer
Owner of product backlog (incl. prioriorities) Decides where the team should go
Not how, not how fast
Scrum Master
Focuses on scrum practices
Removes impediments -> make sure team can work
Team
Team
5-8 persons, sit together Self organizing
Owns sprint backlog
© Sioux 2012 | Confidential | 19
Scrum Artifacts
Product backlog
High level features of project Items at top should be clear Dynamic
Sprint backlog
Contains work for sprint
Fixed for iteration (no changes in current sprint!) Quality more important than features
Visible on scrum-boardVisible on scrum board
Burndown chart
Burndown chart
burndown chart 250 100 150 200 es ti m a te d ho ur s estimated work remaining working hours0 50 10 sept 11 sept 12 sept 13 sept 14 sept 17 sept 18 sept 19 sept 20 sept 21 sept 24 sept 25 sept 26 sept 27 sept 28 sept
days left in sprint
© Sioux 2012 | Confidential | 21
Scrum meetings
Sprint contents meeting
Which items from product backlog are added to sprint
b kl backlog
Product owner decides
Daily scrum
Stand-up meeting
Product owner not always present, but should be
Why Scrum?
Flexibility for business:
After each sprint: After each sprint:
product could be shipped
features can be added/removed on product backlog
and priorities can be changed.
Creates focus
sprint can be overseen
only spend preparation time on things that will be
only spend preparation time on things that will be
done
© Sioux 2012 | Confidential | 23
Release flexibility
Problems in old approach:
Quality level increases slowly during projectQuality level increases slowly during project
Long periods of unstable products
What would you prefer:
Getting a product soon and getting an update when it
is available, or
Wait a long time and get the final product
When to use
Suitable when:
time to market is short
time-to-market is short
desired functionality still uncertain
dynamic market situation
© Sioux 2012 | Confidential | 25
The case against
Requires trust
Too little upfront thinking:
Too little upfront thinking:
Danger of major redesign
Wrong requirements
Requires good team
Scrum and Testing
© Sioux 2012 | Confidential | 27
Test Driven Development
Not part of Scrum, based on eXtreme Programming
But used typically in Scrum projectsBut used typically in Scrum projects 1. Create a new test
2. See the new test fail
3. Write code to make test pass
4. Build, run all tests, see all tests pass, , p
Test Driven Development
Performed by software developer
Tests are continuously executedTests are continuously executed
Code is not delivered when tests fail
Testing is part of the process
Quality awareness increases
Automation is a must!
© Sioux 2012 | Confidential | 29
Test Driven Development
Add tests to regression test
Regression test is run on each buildRegression test is run on each build
Regression detected immediately
Short feedback loop
Safety net at re-designs
Æ Test Engineers should become Software Engineers
or look for another (non-agile) assignment!
Scrum and testing
TDD is important
TDD can be enough in small teamsTDD can be enough in small teams
In combination with continuous integration very strong
But test engineers can still be very useful
Tester ≠ Developer
Add a tester to each scrum team
Tester can define and execute test cases to verify
integrated functionality
© Sioux 2012 | Confidential | 31
Quadrant 1
Technology facing tests that support the team
Unit/component level
Unit/component level
TDD is used here
Focus on internal quality
Instant feedback
Build testability into code
Fully automatedFully automated
© Sioux 2012 | Confidential | 33
Quadrant 2
Business facing tests that support the team
Verify correctness of complete function
Verify correctness of complete function
Integration
If necessary use stubs and mocks
Focus on external quality
Quick feedback (during sprint)
Test engineers together with customersTest engineers together with customers
Manual and automated
Quadrant 3
Business facing tests that critique the product
Focus on validation and acceptance
Focus on validation and acceptance
Customer important
Realistic environment/use
Usability
Quick feedback (end of sprint)
Increases confidenceIncreases confidence
© Sioux 2012 | Confidential | 35
Quadrant 4
Technology facing tests that critique the product
Non functional tests
Non-functional tests
E.g. Performance, reliability, security
Stress- and load testing
Special expertise needed
Tool support needed
Probably not in early sprintsProbably not in early sprints
Role of Test Engineer
Æ Test Engineers should become Software Engineersor look for another (non-agile) assignment!( g ) g
In fact a Test Engineer can add a lot of value
In the Scrum team
Æ Move Test Engineers from independent test teams
into scrum teams
© Sioux 2012 | Confidential | 37
More than only software
Multiple disciplines
© Sioux 2012 | Confidential | 39
Multiple disciplines - Agile
Whish Product Manager
Still magic, but smaller magic
Problems/challenges
Hardware and Software developed concurrently
Development machine not the targetDevelopment machine not the target
Used for testing (e.g. TDD) Can differ substantially
Hardware running ahead of software
Problems in hardware detected late and hopefully fixed in software
Or hardware available late in project
Or, hardware available late in project
Software cannot be tested on target hardware
© Sioux 2012 | Confidential | 41
Problems/challenges
Hard to “scrum” hardware and mechanics
But:But:
There are more deliverables than the final hardware Documents (e.g. interface specifications)
Prototypes
Alternative hardware (competitor, old-version)
Stubs and simulators (software alternative), test interfaces
Remember: we want to reduce risk as soon as possibleRemember: we want to reduce risk as soon as possible
It might be worth to spend a few euro’s on extra prototypes Although in the end, they are “thrown away”
Problems/challenges
In some environments Hardware/Mechanical
development “rather” predictablep p
Choice can be not to disturb them with a new process
Software can still benefit from Scrum
ÆWhole project can still benefit from Scrum
© Sioux 2012 | Confidential | 43
System level
Anyway...
System test activities must still be performedSystem test activities must still be performed
As soon as possible
Targets needed (might be sparse)
In sprint, but often not practical
Take system test out of sprint, and let separate team y p , p
look at this:
Independent Test Team?
System tests Integration tests System tests Integration tests Scrum System tests Integration tests Sprint n Sprint n+1 Integration testsDevelopment & Unit tests
Integration tests Development & Unit tests
Integration tests Development & Unit tests
Integration tests Development & Unit tests
Release Scrum Team: Team: Release Sprint n+2 Integration tests Development & Unit tests
Release
Integration tests Development & Unit tests
Sprint n Sprint n+1 System tests p p Release Independent Test Team: Internal
Release InternalRelease
Sprint n+2 p
Internal Release
© Sioux 2012 | Confidential | 45
Multiple Scrum teams
Multiple Scrum teams
Potentially
Sprint
Contents Sprint Team A
Potentially Shippable
Product
Sprint
Contents Sprint Team B
Potentially Shippable
Product
Sprint
Contents Sprint Team C
Potentially Shippable
Product
© Sioux 2012 | Confidential | 47
Multiple Scrum teams
Sprint
C t t Sprint Team A CSprintt t Sprint Team A Contents Sprint Team A
Potentially Shippable Product Sprint
Contents Sprint Team B
Sprint
Contents Sprint Team C
Contents Sprint Team A
Potentially Shippable Product Sprint
Contents Sprint Team B
Sprint
Contents Sprint Team C
Role of Test Engineer
Æ Move Test Engineers from independent test teamsinto scrum teams
Independent test engineers still needed
Probably less, so some can move to scrum teams
© Sioux 2012 | Confidential | 49
What about regulations?
Satisfying standards, regulations each sprint?
Probably notProbably not...
Mandatory at the end of development (End Game)
Think of interim releases
Overhead can be large (e.g. FDA approval)
But benefit can be highg
Customer feedback
Problems with standards, safety, etc detected early
Æearly risk reduction
Evidence… traceability?
When formal process is needed to release a product
Safetyy
Industry standards
Regulatory approvals (e.g. FDA)
Overall quality characteristics
Best practise:
Can be done outside the Scrum teams
Can be done outside the Scrum teams
By independent teams
Test, regulatory, quality, (hardware) standards, ...
Link to the Scrum teams, but still independent
© Sioux 2012 | Confidential | 51
Scrum and CMMI
Scrum focuses on people and interaction.
CMMI focuses on processes
CMMI focuses on processes.
Scrum focuses on working software.
CMMI focuses on documentation.
CMMI and Scrum can be combined, but their natural
Outsourcing
© Sioux 2012 | Confidential | 53
Common issues with
outsourcing
Difficult communication
Cultural differencesCultural differences
Limited domain and system knowledge
Requirements must be clear and very detailed
Time-zone difference
Not under control
Focus
Scrum and outsourcing?
Some bullets from other sheets: Self-organizing teams
Daily scrum meetings to share status
Requirements are captured as items in a list of “product backlog”
Quick feedback from customer
Self-steering teams
The whole team takes a shared responsibility on achieving the project goal (accountability and authority).
desired functionality still uncertain
Æ Don’t combine Scrum and outsourcing
© Sioux 2012 | Confidential | 55
Sioux and outsourcing
Moscow
Availability of skilled engineersAvailability of skilled engineers
European culture
English speaking
Minimal time zone difference (2 hours, but...)
Frequent travelling possible
Of course: lower hour rate
2 customers are using this service
Wish, need Release
V-Model
Integration Test Architecture SystemRequirements System Test
User
Requirements Acceptance Test
FrontOffice Eindhoven Implementation Component Test Design Integration Test Architecture BackOffice Moscow © Sioux 2012 | Confidential | 57
FO – BO responsibilities
FO Requirements elicitation / discussions with customerq
Architecture (high level)
Project management
BO
Architecture (low level)
Design
Implementation & Test (simulation)
Implementation & Test (simulation)
FO
Integration and system test (targets)
Reporting
Outsourcing and Scrum
Sprint backlog defined by FO+BO and customer
Sprints of 3 weeksSprints of 3 weeks
Daily scrums
Demo’s given by FO to customer
Retrospectives with the team
FO present at customers site (regularly)
Some extra tools needed:
Skype, WebEx, electronic scrum-board
© Sioux 2012 | Confidential | 59
Advantages
Daily scrums overcome multiple problems:
CommunicationsCommunications
Focus
Control
FO-BO construction overcomes:
Domain and system knowledgey g
Requirements
Æ Don’t combine Scrum and outsourcing
© Sioux 2012 | Confidential | 61
Final thoughts
Scrum is not the solution for all problems
You still need good peopleYou still need good people
Scrum is not new
Focus on team work and quick feedback
(Independent) Testing still necessary
Scrum possible in (multi-disc.) product development
Scrum helps when outsourcing is appliedp g pp
Consulteer onze website