Software Process Improvement CMM
Marcello Visconti
Departamento de Informática
Universidad Técnica Federico Santa María Valparaíso, Chile
Software Engineering Institute
♦ Founded by the Department of Defense (USA), at Carnegie Mellon University, in 1984
♦ First director: Watts Humphrey
♦ Goal: to establish standards of excellence in software engineering and accelerate the transition of advanced technology and methods to practice
Software Engineering Institute
♦ Important result: Software Process Maturity Model - developed to evaluate the capabilities of a software organization and identify the most
important improvement areas - considering the complete process of software development as a process that can be controled, measured and improved
Software Engineering Institute
♦ In order to improve their capabilities, the software organizations must: understand the present state of their software process; develop a view of the desired process; establish a list of the improvement actions required, in priority order;
produce a plan to achieve those actions; and commit the resorces to execute the plan
♦ Tool: maturity framework
SEI’s maturity model
♦ 5 level model (presented in 1987)
♦ Each level establishes an intermediate set of goals to achieve the higher level of maturity
♦ “The 5 maturity levels reasonably represent the historical phases of evolutionary improvement at real software
organizations, represent an improvement measurement that is reasonable to reach from the previous level, suggest
improvement goals and intermediate progress measurement, and make obvious a set of priorities of immediate
improvement once the status of an organization with respect to the framework is known” (Watts Humphrey)
SEI’s maturity model
♦
5 levels
> initial (ad hoc/chaotic)
> repeatable (intuitive)
> defined (qualitative)
> managed (quantitative)
> optimizing (feedback)
SEI’s maturity model
♦
Initial level
> lack of formal procedures, cost estimates, project plans, administration mechanisms to assure that the procedures are followed, ill- integrated tools, non-existent change control, higher administration does not understand key issues
SEI’s maturity model
♦
Repeatable level
> process dependent on the individuals, basic project control are established, strengths when facing similar work, high risk when facing new challenges, lacks an ordered framework for improvement
SEI’s maturity model
♦
Defined level
> process defined and institutionalized, existence of a Software Engineering Process Group
(SEPG) established to lead improvement
SEI’s maturity model
♦
Managed level
> process is measured, a minimum set of quality and productivity measurements established, a process database established with resources for analysis and maintenance
SEI’s maturity model
♦
Optimizing level
> improvements fed back to the process, data collection automated and used to identify weakest process elements, numerical evidence used to justify application of technology to critical tasks, rigorous cause analysis and defect prevention
SEI’s maturity model
♦ Assessment procedure: review of an organization software process developed by a professional team, on site, during a week approximately
♦ Main tool: questionnaire with 101 yes/no questions
SEI’s maturity model
♦ Main criticisms
> tool to assess defense contractors (USA)
> score determination (maturity level)
> final reports not detailed enough
> level 1 is reached with no effort
> question scalability
> algorithm complexity
> score determination inflexibility
> yes/no responses
> lack of clarity in levels 4 and 5
SEI’s maturity model
♦
Suggestions
> modify the evaluation method
> de-emphasize maturity level (score)
♦
Answer
> new model: CMM (Capability Maturity Model), in 1991
Capability Maturity Model (CMM)
♦ Decompose each maturity level in key process areas (KPA), key practices, and key indicators
♦ Key process areas: goals to be reached to get to a particular maturity level
♦ Key practices: procedures and activities that contribute to reach the goals
♦ Key indicators: help in the determination of goal satisfaction, they form the basis for assessment procedure
Common Features
Key Practices
Infrastructure or Activities Implementation or Institutionalization Goals
Process Capability
contain
organized by contain indicate
achieve address
describe Maturity Levels
Key Process Areas
CMM structure
Capability Maturity Model (CMM)
♦
Level 1 - KPA
> none
Capability Maturity Model (CMM)
♦
Level 2 - KPA
> requirements management
> software project planning
> software project oversight and tracking
> software subcontract management
> software quality assurance
> software configuration management
Capability Maturity Model (CMM)
♦
Level 3 - KPA
> organizational process focus
> organizational process definition
> training program
> integrated software administration
> software product engineering
> intergroup coordination
> peer reviews
Capability Maturity Model (CMM)
♦
Level 4 - KPA
> process quantitative management
> software quality management
Capability Maturity Model (CMM)
♦
Level 5 - KPA
> defect prevention
> management of technology change
> management of process change
Capability Maturity Model (CMM)
♦ De-emphasizes score (maturity level) out of an assessment - final product is now a profile of key process areas, indicating their level of satisfaction
♦ The maturity level is established as that in which all key process areas are satisfied continuosly
Capability Maturity Model (CMM)
♦ Development of CBA/IPI, CMM-based appraisal for internal process improvement
♦ It has emphasized identification of problems and improvement actions over the numerical score
♦ It lacks conclusive validation; preliminary data are favorable (high ROI)
♦ It has become a de facto standard in the software industry
Capability Maturity Model (CMM)
♦ CMM version 1.0: initial release, 1991-1992
♦ CMM version 1.1: 1993, only version used
♦ CMM version 2: 1996, never used (conflict with development of CMMI)
♦ Framework hasn’t changed, new versions improve the structure of key process areas, practices, indicators, and improve the questionnaire
3 dimensions of CMM
♦ Eyesight: ability to observe, to identify status and progress -- levels 1, 2, and 3
♦ Foresight: ability to aim, to goals and objectives -- level 4
♦ Insight: ability to improve, to take corrective actions, learn, and change -- level 5
CMM early results
♦ Study at Hughes Aircraft Software Engineering Division, 1987-1990
♦ Results of first assessment: level 2
♦ Results of second assessment: solid level 3
♦ First major study in SEI’s validation effort
CMM early results
♦
Estimated costs
> US$ 45.000 in assessment
> US$ 400.000 in training program
> 75 person-month effort
♦
Estimated benefits
> US$ 2.000.000 per year (savings)
> better work conditions, motivation, performance
CMM early results
♦ Study at Raytheon Equipment Division, 1988-1991
♦ Evolution from level 1 to level 3, going through level 2
♦ Improvement generated savings of US$ 8.200.000 in rework reduction, with an investment of US$
1.100.000, and a return of investment of 7:1
CMM early results
♦
Other studies and results (early 90s)
> Lockheed: study predicts that between levels 1 and level 5 quality increments can be 100 times, and productivity increments 10 times
> Motorola USA: similar to Raytheon
> Motorola India: directly to level 5 in 2 years, using level 3 process at Motorola USA as a base
CMM early results
♦
1991
>
59 sites
>
sites
• 81% at level 1 CMM
• 12% at level 2 CMM
• 7% at level 3 CMM
• 0% at level 4 CMM
• 0% at level 5 CMM
CMM early results
♦
1996
>
477 sites
>
sites
• 69% at level 1 CMM
• 18% at level 2 CMM
• 11% at level 3 CMM
• 1.5% at level 4 CMM
• 0.4% at level 5 CMM
http://www.sei.cmu.edu/sema/pdf/2002aug.pdf
State of the practice
♦ A recent case study: Northrop Grumman Electronics, 600+ software engineers
♦ Rated at level 2 (1987) and level 3 (1989)
♦ SPI annual cost: US$2 million
♦ Starting in 1996, two main goals
> accelerate productivity gains
> move to the use of product lines, architecture and systematic reuse
♦ Moved to level 4 by 2001
State of the practice
♦
Tangible benefits
> accelerate productivity gains
• 20% annually, up from 10% anually
• cost avoidance: US$25 million annually
> move to the use of product lines, architecture and systematic reuse
• savings: US$20 million annually
• extra cost: US$800K annual
♦
Estimated ROI - 15:1
State of the practice
CMM Projections
♦ USA: level 3 minimum established by Department of Defense for its contractors (10 years ago)
♦ Relation CMM - ISO 9000: CMM level 3 implies certification ISO 9001? and viceversa?
♦ CMM -> SW-CMM -> CMMI
♦ CMM family - extension to other areas
CMM Family
♦ Software CMM (SW-CMM): construction of software products
♦ People CMM (P-CMM): development of human resources
♦ Software Acquisition CMM (SA-CMM): adquisition of products and software-intensive systems
♦ Trusted CMM (T-CMM): construction of highly reliable software products
♦ Systems Engineering CMM (SE-CMM): construction of products and delivery of services (focus: systems engineering)
♦ Integrated Product Development CMM (iPD-CMM): construction of products and delivery of services (focus: collaboration of all disciplines)
♦ CMM integration (CMMI)
CMMI
♦ CMM Integration
♦ Source models: SW-CMM V2C, SECM, IPD-CMM
♦ Design goals
> integrate source models, eliminate inconsistencies, reduce duplication
> assure consistency with ISO 15504 (SPICE)
> support to introduce new disciplines as needs arise
♦ Developed in DoD sponsored collaboration between industry, US government, SEI
♦ SCAMPI - Standard CMMI Appraisal Method for Process Improvement - similar to CBA/IPI
CMMI
♦ Benefits
> integration of system engineering and software engineering for aditional productivity & quality gains
> efficient, effective assessment and improvement across multiple process disciplines
> a common, integrated vision of improvement for all elements of an organization
> can be used for appraising processes as well as process improvement
CMMI
♦
One model, two representations (both include essentially the same information)
> CMMI Staged
• maturity levels (1-5), each defined in terms of a number of process areas
• organizational characteristic
> CMMI Continuous
• capability levels (0-5), for a process area or a set of process areas
• process or process area capability
CMMI
♦
Maturity vs. Capability - concepts are
similar, but process area capability deals
with a set of processes relating to a single
process area or specific practice, while
organizational maturity pertains to a set of
process areas across an organization
CMMI
♦ Selecting a representation
> CMMI as a means, not done for its own sake
• CMMI and SPI should help people, projects, programs &
organization
> Business goals must guide also the goals that organizations have regarding CMMI
> Advantages of continuous representation
• provides maximum flexibility for focusing on specific process areas according to business goals and objectives
> Advantages of staged representation
• clear goals and proven path for process improvement
• facilitates comparison of different organizations
• a familiar model for CMM users
