Leveraging Agile and CMMI for better Business Benefits Presented at HYDSPIN Mid-year Conference Jun-2014

(1)

Leveraging Agile and CMMI for better Business

Benefits

(2)

2

Outline

• Context

• Key Business Imperatives

• Agile Adoption and CMMI Roadmap

• CMMI+Agile Best Practices & Key Benefits

• Overall Performance Results

(3)

3

Context

• Sonata’s portfolio of services include: Product Engineering Services to ISVs,

ADM, Managed Services, Testing, BI & Analytics, IMS and SMAC

• Sonata has a strong quality management system evolved through more

than two decades by aligning & adopting to various international standards

and models such as ISO9001, CMMI, ISO27001 and ISO2000-1

• After the CMMI appraisal in Apr 2011, there were various triggers driving

the process improvement initiatives

• Voice of customer

– Significant increase in the number of customers looking for faster and high quality delivery at an optimum cost

(4)

4

Context…

• Voice of the Process

– Process Performance Baseline Report May’12 – Process Compliance & Effectiveness Analysis

• Portion of non-compliances due to QMS not aligned to Agile Delivery Methods

• Voice of the Business

– Adopting CMMI DEV v1.3 and getting assessed at Maturity Level 5

– Enhance the QMS aligning to CMMI DEV v1.3 and Agile Delivery Methods

(5)

5

Key Business imperatives

Faster time to market

Swift responsiveness to changing business needs

Better visibility, control and collaboration

Improved Product Quality & Resilience

Intolerance to waste and overheads

Ownership

and Accountability

(6)

6

Agile Adoption with CMMI DEV v1.3 Roadmap

People Enablement

Agile Training to Scrum Masters & Team Members Agile orientation to customer’s teams wherever required

Establishing processes, guidelines, templates to make Agile projects compatible to CMMI 1.3 Tools evaluation and recommendation

Process Awareness and Orientation Sessions for Agile

Establish PPBs at each ODC level based on past data

Develop PPM for predicting performance at Sprint & Release level

Evaluating the progress periodically and fine-tune the processes

(7)

7

(8)

8

(9)

9

Project Planning

Based on one line requirement E0 • Single page business need • Project Brief • Discussions E1 (Gate 1) • High level requirements • Solution blueprints E2 (Gate 2) • Detailed Solution design – UX and application • Wireframes E3 (in Agile Sprints) • Discussions between business, architecture, IT solutions (Customer Technical Team and Sonata)

• High level estimates with assumptions and dependencies • +/- 50% variations

estimated from actuals

• Estimates at individual requirements / user story level

• Based on solution with assumptions on detailed integration between components • +/- 30% variations

estimated from actuals

Epic level User Story

level

Task level

• Estimates based on technical tasks to complete a user story

• Sized in story points • Productivity based on

velocity of the team normalised across projects • +/- 10% variations estimated

from actuals

• Reviewed and validated by Customer Technical Team • Based on one liner

(10)

10

Two Week Sprint

(11)

11

CMMI+Agile Best Practices (1 of 3)

CMMI+Agile Best Practices Key Benefits

Release Planning

 Use PPB and PPM for arriving at Release Schedule for a Minimum Viable

Product(MVP)

 Use PPB and PPM for deciding the Team Resource Mix and Capacity

 Time Boxed Sprints

 Realistic Schedule Forecast  Improved Capacity and

Team Planning

Sprint Planning

 User Story Walkthrough by Product Owner  Solution Blueprint Walkthrough, if

applicable

 Use PPM for Effort Prediction and decide how much user stories to commit

 Use PPB & PPM for optimum assignment of user stories based on complexity

 Requirement Clarity and Completeness

 Improved Sprint Planning

(12)

12

CMMI+Agile Best Practices (2 of 3)

Sprint Execution

 Requirement Traceabilty through tool

 Use of Decision Analysis techniques for better Design and Project Management

 Use of Code Quality tools such as Sonar, Resharper etc.

 Sub-process monitoring for controllable metrics such as Coding Delivery Rate, Code Complexity, Test

Coverage etc.

 Refactoring

 Automated Unit Testing using Junit, Nunit etc.  Continuous Integration

 Show and Tell Demos

 Automated Build and Deployment

 Reduced Process Overhead

 Improved Product Quality

(13)

13

CMMI+Agile Best Practices (3 of 3)

Release and Sprint Retrospective

 Use of Causal Analysis and Resolution techniques

Effective Learning & process

improvements Organizational improvement beyond the project team

 Organizational Process Areas such as Organizational Training, Process Focus, Definition, Process

Performance, Performance Management

 Innovations implemented through OPM process resulting in significant improvement in delivery  Organization Knowledge and Process Asset

Repositories  Holistic approach to Agile/Lead Practices Adoption  Risk reduction through process standardization, people capability development and innovations

System’s View and Product Integration

 Use of Product Integration approach and practices

 Robust Product Integration

(14)

14

Framework for Quantitative Project Management

ODC1 PPB

ODC1 PPM

Effort Prediction(Big Y): a) Developer Skill (X1) b) Designer skill (X2)

Defect prediction

1. Junit test coverage 2. Unit testing delivery rate 3. Class complexity level ODC Business Objectives

(GM & CSAT) Project QPPOs (Productivity and Defect Density) Identify critical sub process (sensitivity analysis)-Coding Delivery rate , Testing Defect Density PPM Composition:

List of independent variables: 1)Delivery rates at sub-process level

a) Product Backlog Analysis Delivery rate-Distribution b) Design Delivery

rate-c) Coding Delivery

rate-Developer and Designer Skill-Expert/Medium/Novice –Distribution

d) Code Review Delivery Rate-Distribution e) Unit Test Scripting and Testing Delivery

Rate-Distribution

f) Test Case Design, Test case Review Delivery Rates g) Manual QA Testing Del Rate

h)Automation Scripting, Automation Script Review, Automation Testing Del Rates

i) Testing Defect Density

2)Size of user stories (Sprint)– Story Point 3) Available Capacity or Revised Capacity

Identifying Risks in completing the Iteration within available capacity

Control Critical Sub process (Control Chart)

Do CAR (if any deviations)

Change /Improve process

(15)

15

• Purpose

– To assess the certainty of completing the committed product backlogs in an iteration within the available capacity. The prediction is done based on the historical

performance from past iterations on each sub-process delivery rates from PPB.

• Simulation Model predicts the total effort based on the following inputs

– Size of Product Backlog Items

– In-scope of sub-process tasks for Sonata offshore team – Skill of Developer and Designer

– PPB Values for sub-process delivery rates

• Other inputs that will influence the certainty of completing the committed backlog items in an iteration include

– Initial Capacity (based on Team Size, Ideal hours per day, Number of working days and exclusions due to leaves and holidays)

– Revised Capacity (based on Revised Team Size (addition of shadow or backup resources), Revised Ideal hours per day, Revised number of working days and exclusions due to leaves and holidays)

Purpose of Prediction Model in an ODC

(16)

16

• Sprint 8 has 72 story points in scope with 145 days of available capacity.

• Simulation model was used to predict the certainty of completing 72

story points in 145 days.

– The result of initial run predicted 87% certainty as this was a good certainty there were no “What-If” scenarios established.

– But it was decided to rerun the model mid sprint, after 7 days wherein the certainty was seen to be 100%.

Effort prediction Model Usage during Sprint 8 Planning.

(17)

17

Effort prediction Model Usage during Sprint 8 Planning

17

From the output of the prediction model, it is inferred that it is 87% certain that all the planned backlog items(72 story points) in Sprint 8 can be completed with 145 man days of effort, which is the team capacity. As the certainty level is more than 75%, what-if analysis was not done.

(18)

18 Parameter being predicted Project Objective Input parameters used for prediction Type of prediction model used Prediction Range/Value Certainty/ Confidence Level of prediction Risk assessment based on prediction results and mitigation plan/s implemented Effort for Iteration Completion Story Delivery Rate * Size of user Story * In-scope of Sub-process tasks for Sonata offshore team * Skill of Developer Simulation Model 145 man days 87% Minimal risk

Effort PPM Usage during Sprint Planning

(19)

19

• The prediction model was rerun after 7 days into Sprint by providing

actual effort spent so far for each of the user stories.

• It was observed that there is 99% certainty after 7 days.

• This confirmed that the scope of the sprint will be met with the available

capacity and velocity of the team was also good which provided 99%

certainty within 7 days into sprint.

PPM Usage during mid Sprint

(20)

20

Effort prediction Model Usage during Sprint 4 Planning

20

During mid sprint with 7 days effort we get 99% certainty.

(21)

21 Parameter being predicted Project Objective Input parameters used for prediction Type of prediction model used Prediction Range/Value Certainty/ Confidence Level of prediction Risk assessment based on prediction results and mitigation plan/s implemented Effort for Sprint Completion Story Delivery Rate Actual Effort of tasks completed after 5 days into sprint. Simulation Model 145 99% No risk

Effort PPM Usage during mid Sprint

(22)

22

• Defect prediction using PPM is done through simulation

model.

• The inputs that control defect prediction are as below

– Size of the sprint in story points

– Resource level skill assigned to each user story.

Defect prediction using PPM

(23)

23

Defect prediction PPM during planning

23

For a target SIT defect density of 0.3 and current

assignment of user stories predicted a certainty of 100%

From the model it has been identified that skill level of resource assigned to each user story has major impact on defect density. Varying the skill level of the resources between user stories is an option that could be used to control defects. Apart from this the sonar parameters are

(24)

24

Agile Adoption Statistics @ Sonata

Detail FY12 FY14

Percentage of Agile projects 20% 60%

Application Development and Maintenance 1 customer 4 customers Product Development and Engineering services 1 customer 3 customers

Testing and Migration 1 customer

Number of Agile engagements initiated during the year 25% 75% Distribution of current Agile projects based on location

Leveraging Agile and CMMI for better Business Benefits Presented at HYDSPIN Mid-year Conference Jun-2014