10,000 Miles away: Developing with a Distributed Team

(1)

Or, “How to be Agile at a Distance”

Instructor: Kevin Thompson, PhD, PMP, ACP, CSP, CSM

4100 E. Third Ave, Suite 205, Foster City, CA 94404 | 650-931-1651 | www.cprime.com

10,000 Miles away: Developing

with a Distributed Team

(2)

Who is cPrime?

(3)

Today’s Presenter

Kevin Thompson, Ph.D.

Agile Practice Lead

• Trains, coaches teams and companies in Agile development

• Education and certifications

 Certified ScrumMaster and Scrum Professional

 PMI Project Management Professional

 PMI Agile Certified Practitioner

 Scaled Agile Framework Program Consultant

 Certified Yellow Belt Collaboration Architect

(4)

Congratulations!

You are VP of Engineering for a small software company with big ambitions, located in Palo Alto, California.

Your first product is GloCAD.

 GloCAD enables mechanical engineers around the world to collaborate in the production of complex electro-mechanical designs

You need to build a world-class software-development organization. You are going to start with one, small Team, and grow over time. You have selected the

Scrum process to provide the flexibility, visibility, and short time-to-market the company needs.

(5)

What We Need to Know First



What does “Agile” mean?



What is an Agile process?

(6)

(7)

What is an Agile Process?

In principle:

Any process that adheres to the principles of the Agile Manifesto

“We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.”

Manifesto for Agile Software Development, www.agilemanifesto.org

The concepts arose in software project management, BUT…

(8)

Adaptive Spectrum

Drives

Process Selection

All processes have their “sweet spots”

Based on scope, effort uncertainty

Predictive Adaptive Reactive

Plan-Driven Scrum Kanban Waterfall XP

SDLC

PREDICTIVE REACTIVE

The Agile Zone

Adaptive processes

• Emphasize adaptability to

Reactive processes

• Don’t require planning

Predictive Processes • Emphasize Efficiency • Perform poorly when

(9)

Key Scrum Concepts

Product Owner provides ranked requirements, as short narrative

descriptions (“Stories”), or bug-fix requests. Set of

unscheduled requirements is the “Product Backlog.” Each

requirement is a Product Backlog Item (PBI).

Small Teams (3—9 people) work in short “Sprints” (2—4 weeks)

to implement stories in rank order.

• Requirements frozen when Sprint starts—no change requests allowed!

Teams self-organize to best apply member skill sets (coding,

test development, testing, etc.).

• ScrumMaster does not assign tasks

ScrumMaster does whatever is needed to make Team as productive as possible

• Focuses on planning, tracking, mentoring, resolving issues, enforcing process

(10)

Agile Collaboration by Swarming

How many people can work on Story #1?

They swarm on #1.

How many people can work on Story #2?

(11)

Tracking

• Scrum Taskboard shows plan, tracks status of work in Sprint

• E.g., write Stories, Tasks on sticky templates, put on

Taskboard

• Team members move Tasks to update status

(12)

Scrum Ceremonies

Ceremony Time

Box Input Output Value

Backlog

Grooming* <1 hr

Draft User Stories, Epics from Product Owner

Finalized User Stories Technical Stories Ranking for top PBIs

Product Backlog & Team are ready for Sprint Planning Sprint Planning 2 - 8 hr Ranked Product Backlog with Acceptance Criteria Sprint Backlog:

• Selected stories + estimates

• Tasks + estimates

Team has a plan to implement Sprint Backlog

Daily Scrum

(Stand-up) <15 min In-progress Tasks

Tasks updated

Impediments raised

Team members on same page re: Sprint progress and impediments

Sprint

Review < 1 hr

Demo prepared for completed stories

New Stories, based on review by Product Owner

Ranking may be revised

Deliverables reviewed; feedback from stake-holders, other teams

(13)

Sprints Repeat in a Cadence

• Cadence: Rhythmical motion or activity

• Requirements are completed before Sprint starts • Planning is continuous, not phased

Working Days Day 1

Backlog Grooming

Day 31 Sprint Planning Meeting

Create Task Breakdowns Begin Dev & Testing

Sprint Review

Day 3 Day 5 Day 7 Day 9 Day 11 Day 13 Day 15 Day 17 Day 19 Day 21 Day 23 Day 25 Day 27 Day 29

Sprint N-1 Sprint N Sprint N+1

Retrospective

Create Task Breakdowns Begin Dev & Testing

Sprint Planning Meeting Backlog

Grooming

Backlog Grooming

(14)

(15)

Starting out with the First Scrum Team

• This Scrum Team has

• One experienced ScrumMaster

• One inexperienced but committed Product Owner

• Seven skilled Team members (5 developers, 2 QA specialists)

• This Team

• Plans one Sprint at a time

• Tracks progress with a physical Scrum Taskboard

 Stories & Tasks on sticky notes

 States are Not Started, In Progress, or Complete

 ScrumMaster draws daily Burndown chart by hand

• Is serious about test automation, using

 JUnit for unit and integration testing

 Cucumber for acceptance testing

 Selenium for UI-level testing

(16)

How the First Few Sprints Go

• Most of Sprint 1 focuses on setting up tools and

environments for development and testing

• Estimates are very poor • Tracking is erratic

• The plan (Sprint Backlog) and reality differ substantially

• Sprint 2 gets more done on the product

• Estimates are better

• Visibility into status is improving • Reality resembles plan

• Sprints 3 and on improve

• Estimation and planning improve

(17)

Time to Grow

• Funding from early-stage investor will help us move faster

• We hire seven more people, for a total of 14

• Maximum Team size in Scrum is 9, so

• Organize into two Teams

• New Team moves into building next door • Have same ScrumMaster for both Teams • Have same Product Owner for both Teams

• We’ve clearly mastered Scrum

• We’re ready for growth

• Nothing will go wrong now, right?

(18)

(19)

Here’s Why

• Team A changes an interface.

• Team B’s code stops working

• Guess who else used that interface?

• Team B deletes some obsolete code they don’t need any

more

• Team A’s code stops working

• Maybe that code wasn’t obsolete after all

• We have a new kind of problem: Cross-Team impacts

(20)

How we

Implement

Cross-Team Requirements

• Cross-Team “Scrum of Scrums” meetings

• Purpose: Identify & address cross-Team issues

• Frequency: As needed (daily, semi-weekly,…)

• Participants:

• Facilitator (e.g., Program Manager) • Representative from each Team

 Team Member, ScrumMaster

• Agenda:

• Each person describes

 What my Team is doing that may affect other Teams

 What issues my Team needs help to resolve

• Resolve issues in meeting, if possible • Identify follow-up actions and owners

(21)

Adding Scrum of Scrums Meeting Helps

• Twice-weekly forum with representative from each Team

ensures cross-Team issues are addressed

• It becomes clear that a major contributing factor to

cross-Team issues is lack of visibility for each cross-Team into the status of work, plans for the other Team

• They use paper-based Scrum Taskboards • They are in different buildings

• They can’t see each other’s Taskboard

(22)

Introduce Web-based Agile PM Tool

• Agile Project-Management tools provide global visibility for

requirements, plans, and status of work

(23)

Scrum Metrics for Tracking Progress

(24)

Better, but…

• Plans, status of work are visible to everyone.

• Easier to see what each Team is doing that might affect other Teams

• Fixing this problem clarifies the existence of another one:

• It’s getting hard to find documents other than requirements

• API (Application Programming Interface) definitions • User Interface Style Guides

• Coding standards

• Design, Architecture documents

• These are relevant for all Teams. Emailing them on request

isn’t satisfying need for availability, and it doesn’t make sense to attach them to Stories in Jira.

(25)

Provide Repository for Information

• Use this for everything not tied to a specific Story

• Wikis, SharePoint, Basecamp

(26)

Now Teams are Running Smoothly

• We’ve overcome our growing pains

• Our multiple-Team environment is developing products

smoothly, reliably

• We’re handling cross-Team issues effectively

• Everyone can see status, plans, and information of general

interest

• We’re set, now.

• Ready for future growth.

(27)

Time to Grow—Again

• We’ve shipped our first release of GloCAD!

• Sales are good, demand is growing

• We hire 14 more people, for a total of 28

• Maximum Team size in Scrum is 9, so

• Organize into four Teams

• Move everyone into a bigger office

• Have same ScrumMaster for all four Teams • Have same Product Owner for four Teams

(28)

What Happens Next

• Everything seems fine, at first

• Then we start to notice problems

• Discipline deteriorates

• Planning becomes haphazard

• Some Task Breakdowns are late or missing for some Stories • Teams end Sprints with some Stories partially completed

• Teams aren’t getting enough User Stories

• Product Owner can’t keep up with demand • Teams make up things to do

 Nice opportunity to keep technical debt low, but

 Not the best use of time and resources

• Product Owner is overworked

(29)

Solution

• A ScrumMaster or Product Owner cannot support more

than three Teams!

• Introduce a second of each

• Now each ScrumMaster and Product Owner supports two Teams • Best to pair them: Same SM and PO for a set of Teams

• Some things improve

• Discipline, effectiveness, quality improve

(30)

Problems

• Other problems become clear

• Teams need things from other Teams that haven’t been

developed

• Confusion, delays, and disruption result

• Marketing department is frustrated at inability to get

advance notice of new features in time to develop marketing campaigns

(31)

Create a Release Plan every Three Months

• A Release cycle is a set of Sprints

• Release Planning produces a Release Plan

• Estimates for Stories, Epics in Release • Rough map of Stories to Sprints

• Dependencies between Stories, Teams

• Release Planning can be expensive

• 1—3 days for all Teams and members

• Value of Release Planning must justify the

cost

• Reduction in confusion due to planning cross-Team dependencies

• Necessary to understand how external resources may be engaged

(32)

Set up for Release Planning

• Schedule Release Planning Meeting

• Before start of Release cycle • As late as possible

 Typical: 1—4 weeks in advance

• Allow enough time (1/2 – 3 days)

• Publish Agenda

• Prepare for Meeting

• Product Owners: Print Stories, Epics on paper or sticky notes

• ScrumMasters: Estimate Team Velocity for each Sprint in Release • Others: Create presentations, etc.

(33)

How to do Release Planning

1. Have introductory presentations

2. Set up Release Planning board or table

1. Each Team has its own row

2. Sprint boundaries are drawn vertically

3. Conduct Release Planning session (2—4 hours)

1. Program Manager, Product Owners, ScrumMasters assist as needed

2. Teams lay out Stories, Epics (“Items”) in their preferred sequence 3. Teams estimate Items not yet estimated

4. Teams map Items to Sprints

 Stories do not cross Sprint boundaries, but Epics may

5. Teams collaborate to identify, sequence dependencies

 Show dependencies with lines, masking tape, etc.

6. Teams collaborate to identify missing Items, create, and incorporate them into the plan

(34)

The Flow of Release Planning

A1 A2 A3 A4 A5 A6 A7 A8 B1 B2 B3 B4 C1 C2 C3 B5 B6 B7 B8 B9 B10 C4 C5 C6 C7 C8 C9 C10 C11 EB1 EA1

(35)

Example Release Plan

Shows

 Team capacity / Sprint

 Stories per Team per Sprint

 Story sizes

 Dependencies

 Arrows, highlighting Release Name: 3.1

Goal: Fulfill Orders Start: 1-Jan-2012 End: 31-Mar-2012

Sprint A B C

1

Capacity 16 Capacity 26 Capacity 21

Story A1 3 Story B1 13 Story C1 13 Story A2 8 Story B2 5 Story C2 5 Story A3 5 Story B3 8 Story C3 3

2

Capacity 16 Capacity 21 Capacity 26

Story A4 3 Story B4 3 Story C4 8 Story A5 8 Story B5 13 Story C5 5 Story A6 5 Story B6 5 Story C6 13

(36)

Now Teams are Running Smoothly

• We’ve overcome our growing pains—again

• Our multiple-Team environment is developing products

smoothly, reliably

interest

• We’re planning dependencies and setting longer-term

expectations for other departments

(37)

Time to Grow—Again

• We are kicking off our second product!

• BasRelief operates 3D printers to fabricate designs created

in GloCAD

• Integrates with GloCAD • Customer often buy both

• We grow to 60 Team members

• Maximum Team size in Scrum is 9, so organize accordingly

• GloCAD

 Five Teams, two ScrumMasters, two Product Owners

• BasRelief

(38)

New Problems Arise

• Product Owners are overstretched

• Need to write Stories for Teams, work with customers to identify needs and solutions, and negotiate with each other about trade-offs for development

• Cross-Team impacts become overwhelming

• Many Teams means many more cross-Team dependencies

 N Teams have 𝑁 × (𝑁 − 1)/2 interfaces: Grows as 𝑁2.

• Scrum of Scrums helps, but isn’t enough

• Release planning helps, but is taking much longer

• We miss a key deadline

(39)

New Role: Area Product Owner

Split Product-Owner responsibilities across two Roles

Introduce one Area Product Owner per Product

• APO meets with customers to understand needs, solutions

• APO meets weekly with Team Product Owners to build understanding of what needs to be done

• APO provides guidance about tradeoffs • APO participates in Release Planning

• Team Product Owners focus on writing Stories, collaborating with Teams, and not on gathering Customer requirements

(40)

How we

Develop

Cross-Team Requirements

• Product Owner “Scrum of Scrums” meetings

• Purpose: Develop requirements across Teams

• Frequency: As needed (weekly,…)

• Participants:

• Area Product Owner (facilitator) • Team Product Owners

• Agenda:

• Each Team Product Owner describes

 What I’ve done since last meeting

 What I plan to do by the next meeting

 What issues I need help to resolve

• All review status of Release work to date (Burn-Up Chart!), make scope-change decisions

(41)

New Role: Program Manager

Introduce one Program Manager per Product

• PgM facilitates Scrum of Scrums

• PgM facilitates Release Planning, records Release Plans

• PgM tracks, manages cross-Team dependencies to ensure they are satisfied

• PgM removes obstacles that impact multiple Teams • PgM acts like ScrumMaster for a set of Scrum Teams

(42)

Now Teams are Running Smoothly

• We’ve overcome our growing pains—again

• Our multiple-Team, multiple-Product environment is developing

products smoothly, reliably

interest

(43)

Time to Grow—Again

Changes

• Introduce new Area Product Owner, Program Manager for new Teams

• Introduce Release Planning to Philadelphia office • APO’s and PgM’s meet bi-weekly to identify,

negotiate cross-product dependencies

 Decisions flow into Release Planning

• Investment in Jira, Confluence becomes even more

• We buy a small company in Philadelphia, Pennsylvania • They make a key add-on for GloCAD

• They have three Scrum Teams, with one SM and one PO

• Our California-based Scrum Teams need to collaborate with new people in Philadelphia

(44)

(45)

New Problems: Hard to Collaborate at a

Distance!

From the Principles behind the Agile Manifesto:

We cannot collaborate at all until we solve some key problems!

Business people and developers must work together daily throughout the project.

The most efficient and effective method of conveying information

to and within a development team is face-to-face conversation.

Agile processes promote sustainable development. The sponsors,

developers, and users should be able to maintain a constant pace indefinitely.

(46)

Why Co-Location is Preferred

• Members work together easily throughout day

• Proximity encourages interaction • Information propagates rapidly

• Communication is Osmotic

• Members absorb information from questions, answers in background

• Members can chime in if they have something to contribute

• Agile projects favor in-person communication over

documentation

• Co-location encourages and enables good communication • Distribution impairs it, requires more documentation

(47)

The Team Room: What Co-Location Looks Like

Shown

 Protected work area

 Room for Daily Stand-up  Large whiteboard  Information Radiators (Sprint Status)  Comfortable furniture Not Shown  Projector  Large monitors Avoid  Speakerphones  Through traffic

(48)

Distributed Teams: Best Case

• Scrum Teams are

distributed

• Scrum Team members are

co-located per Team

• Compared to total

co-location

• Intra-Team communication is the same

• Cross-Team communication somewhat more difficult, but not hard

• Cross-Team work synced

(49)

Solutions for Collaboration

• Hold distributed meetings during joint working hours

• 9 AM - 2 PM PST, 12 Noon - 5 PM EST

• Use video (Skype, GoToMeeting, WebEx, etc.) for all distributed

meetings (e.g., Scrum of Scrums), wherever possible.

• Use phone or equivalent only when video is not possible.

• Use chat (Skype, HipChat, etc.) for individual real-time Q&A

between Team members, others, across the country

• Use email when real-time communication is not needed, or not

(50)

Now Teams are Running Smoothly

• We’ve overcome our growing pains–again

• Our multiple-Team, multiple-Product, distributed environment is

developing products smoothly, reliably

interest

(51)

Time to Grow—Again

• We buy a small company in Dallas, Texas

• They make a key add-on for BasRelief

• New challenges encountered with the new people

• They have one very informal development team, with no Scrum Roles

• The development people are very junior

• Their code quality and design are poor and won’t scale well • They have much “technical debt” because they’ve never

implemented automated testing

• It is important to improve the add-on’s code quality,

design, and test automation now, before we add many new features

(52)

How We Organize

• Create a new Scrum Team, containing Dallas people, with

focus on building their technical expertise and paying down the technical debt ASAP

• Five Dev and QA people in Dallas

• Two senior developers, one test-automation expert in Palo Alto • Product Owner in Dallas

• ScrumMaster in Palo Alto

• Introduce collaboration practices

• All Team meetings held in common working hours

 9 AM - 3 PM PST, 11 AM - 5 PM CST

• All Team meetings use video

• Scrum Ceremonies conducted as usual, but via videoconference • Scrum of Scrums, Release Planning conducted as usual, via

(53)

(54)

How Well do They Perform?

• Things go well in Palo Alto

• Things do not go well in Dallas, where Team members

• Often do not update Task status in Jira, so we can't track progress • Work on things that have low priority, and don't work on things that

have high priority

• Do not collaborate to complete Stories quickly (with each other or Palo Alto members), but default to having one Team member work on each Story

• Often come late to, or fail to participate in, our key Scrum meetings

(55)

Solution

• Introduce new Role: The ScrumMaster Proxy (SMP)

• ScrumMaster is in Palo Alto

• Does usual ScrumMaster things

• ScrumMaster Proxy is in Dallas

• Does 80% of what a ScrumMaster does, for the people in Dallas

 Pays close attention to who is doing what

 Redirects people to the right things when they are focusing on the wrong things

 Enforces our process and policies (including Task-status updates to enable tracking)

 Removes obstacles to Team productivity

• Has daily synchronization call with ScrumMaster in Palo Alto

• On-site presence of the SMP improves effectiveness of

(56)

(57)

Now Teams are Running Smoothly

• Our multiple-Team, multiple-Product, distributed

environment is developing products smoothly, reliably

• We’re handling cross-Team and distributed intra-Team

issues effectively

interest

(58)

Time to Grow—Again

• New challenges encountered with the new

people

• They use a Waterfall process

• 15 Developers are in New York. 8 QA people are in Shanghai.

 Twelve time zones apart!

• No possibility of changing distribution of people soon

• We buy a company with offices in New York, NY, and Shanghai, China

• They make a solid-modeling program called Densify

• Densify performs physical simulations of GloCAD designs to validate them and reduce need for building costly prototypes

(59)

What we do Next

• Train new people in our Scrum process

• Define three Scrum Teams

• Each has a product-area focus

• Each Team has some NY developers and Shanghai QA people • Two ScrumMasters, two Product Owners in New York

• Program Manager, Area Product Owner in New York • Two ScrumMaster Proxies in Shanghai

• Plan, launch Scrum process

(60)

How Well do the New Teams Perform?

• Growing pains are expected

• Some get better

• Some do not

• Scrum Team members, ScrumMaster, and Product Owner cannot sustain overlapping working hours, for meetings or real-time

collaboration.

 Initial attempt to do this fell apart quickly. Impact threatened burnout and possible loss of people who may quit and look for other jobs.

• Team members in Shanghai, who focus on QA work, cannot get clarity on requirements from the Product Owner when they need it (now), and have at best a one-day response time to their

questions.

 If questions lead to more questions (often the case), resulting back-and-forth email messages may take several days to provide the

(61)

Solution

• Introduce new Role: The Product Owner Proxy (POP)

• Product Owner is in New York

• Does usual Product Owner things

• Product Owner Proxy is in Shanghai

• POP gives real-time guidance about requirements to local people • PO and POP write daily summary of decisions, actions for each

toher

• PO and POP have daily 15-minute call to synchronize understanding

• POP may be wrong sometimes

 Quick answers that are right 80% of the time are better than perfect answers that take one or more days to get

(62)

(63)

Results

• On-site presence of the POP improves productivity, morale

of Shanghai office dramatically

• …but splitting Teams this way will always be costly

• More documentation, less real-time communication

• Latency issues (time from question to answer) can be minimized, but will always be significant

• Impact on productivity, quality of life likely to outweigh perceived cost-reduction benefits of “junior offshoring model”

(64)

In Summary

• Introduce Scrum of Scrum, Release Planning, Area Product

Owner, Program Manager as organization grows

• Don’t split Scrum Teams, if at all possible

• If organization is distributed, keep Teams co-located in different

offices

• This is a very reasonable and effective strategy

• If you can’t keep logically-organized Scrum Teams co-located

• Always have a ScrumMaster Proxy for each Team fragment not co-located with the ScrumMaster

• Consider providing a Product Owner Proxy if Product Owner cannot supply rapid turnaround of questions for remove Team members

(65)

Distributed Teams: Best Practices for Meetings

Working Time Sprint Planning Daily Stand-Up Sprint Review Retro-spective Comment

Full overlap All attend All attend All attend All attend Co-located

Full overlap All attend All attend All attend All attend Designate

one SM proxy (SMP) always, one PO Proxy (POP) as needed, per location for distributed Teams Partial overlap

All attend All attend All attend All attend

Adjacent All attend All attend All attend All attend

Far apart All attend Sub-groups,

SMPs, POPs meet. SMPs, SM provide all findings to full Team. Sub-group nearest to PO demos Sub-groups, SMPs, POPs meet. SMPs, SM provide all findings to full Team. And / Or: Full Team meets twice And / Or: Rotate demo to PO

(66)

Common Reporting Structures - 1

(67)

Common Reporting Structures - 2

PgM: Program Manager USM: Uber ScrumMaster PMO: Project / Program

Management Office

APO: Area Product Owner CPO: Chief Product Owner PMM: Product Management &

(68)

Final Thoughts on Geographic Distribution

• Geographic distribution is not a good way to do Scrum

• …because it is not a good way to do work in general

• Scrum is a good way to do geographically-distributed software

development

• No advantage to other processes, since all processes suffer similarly when spread around the globe

(69)

Now Teams are Running Smoothly

• Our multiple-Team, multiple-Product, globe-spanning

environment is developing products smoothly, reliably

• We’re handling cross-Team and distributed intra-Team issues

effectively

interest

(70)

Time will Tell

www.cprime.com