Process Maturity Framework and Quality Standards

Regardless of which process is used, the degree to which it is implemented varies from organization to organization and even from project to project. Indeed, given the framework of a certain process model, the development team usually defines its specifics such as implementation procedures, methods and tools, metrics and measurements, and so forth. Whereas certain process models are better for certain types of projects under certain environments, the success of a project depends heavily on the

implementation maturity, regardless of the process model. In addition to the process model, questions related to the overall quality management system of the company are important to the outcome of the software projects.

This section discusses frameworks to assess the process maturity of an organization or a project. They include the SEI and the Software Productivity Research (SPR) process maturity assessment methods, the Malcolm Baldrige discipline and assessment processes, and the ISO 9000 registration process. Although the SEI and SPR methods are specific to software processes, the latter two frameworks are quality process and quality management standards that apply to all industries.

2.8.1 The SEI Process Capability Maturity Model

The Software Engineering Institute at the Carnegie-Mellon University developed the Process Capability Maturity Model (CMM), a framework for software development (Humphrey, 1989). The CMM includes five levels of process maturity (Humphrey, 1989, p. 56):

Level 1: Initial

Characteristics: Chaotic�unpredictable cost, schedule, and quality performance.

Level 2: Repeatable

Characteristics: Intuitive�cost and quality highly variable, reasonable control of schedules, informal and ad hoc methods and procedures. The key elements, or key process areas (KPA), to achieve level 2 maturity follow:

Requirements management

Software project planning and oversight Software subcontract management Software quality assurance

Software configuration management Level 3: Defined

Characteristics: Qualitative�reliable costs and schedules, improving but unpredictable quality performance. The key elements to achieve this level of maturity follow:

Organizational process improvement Organizational process definition Training program

Addison Wesley: Metrics and Models in Software Quality Engineering, Second Edition2.8 Process Maturity Framework and Quality Standards

Integrated software management Software product engineering Intergroup coordination Peer reviews

Level 4: Managed

Characteristics: Quantitative�reasonable statistical control over product quality. The key elements to achieve this level of maturity follow:

Process measurement and analysis Quality management

Level 5: Optimizing

Characteristics: Quantitative basis for continued capital investment in process automation and improvement. The key elements to achieve this highest level of maturity follow:

Defect prevention Technology innovation Process change management

The SEI maturity assessment framework has been used by government agencies and software companies. It is meant to be used with an assessment methodology and a management system. The assessment methodology relies on a questionnaire (85 items in version 1 and 124 items in version 1.1), with yes or no answers. For each question, the SEI maturity level that the question is associated with is indicated. Special questions are designated as key to each maturity level. To be qualified for a certain level, 90% of the key questions and 80% of all questions for that level must be answered yes. The maturity levels are hierarchical. Level 2 must be attained before the calculation for level 3 or higher is accepted. Levels 2 and 3 must be attained before level 4 calculation is accepted, and so forth. If an organization has more than one project, its ranking is determined by answering the questionnaire with a composite viewpoint�specifically, the answer to each question should be substantially true across the organization.

It is interesting to note that pervasive use of software metrics and models is a key characteristic of level 4 maturity, and for level 5 the element of defect prevention is key. Following is a list of metrics-related topics addressed by the questionnaire.

Profiles of software size maintained for each software configuration item over time Statistics on software design errors

Statistics on software code and test errors

Projection of design errors and comparison between projected and actual numbers Projection of test errors and comparison between projected and actual numbers Measurement of design review coverage

Measurement of test coverage

Addison Wesley: Metrics and Models in Software Quality Engineering, Second Edition2.8 Process Maturity Framework and Quality Standards

Tracking of design review actions to closure Tracking of testing defects to closure

Database for process metrics data across all projects Analysis of review data gathered during design reviews

Analysis of data already gathered to determine the likely distribution and characteristics of the errors in the remainder of the project

Analysis of errors to determine their process-related causes Analysis of review efficiency for each project

Several questions on defect prevention address the following topics:

Mechanism for error cause analysis

Analysis of error causes to determine the process changes required for error prevention Mechanism for initiating error-prevention actions

The SEI maturity assessment has been conducted on many projects, carried out by SEI or by the organizations themselves in the form of self-assessment. As of April 1996, based on assessments of 477 organizations by SEI, 68.8% were at level 1, 18% were at level 2, 11.3% were at level 3, 1.5% were at level 4, and only 0.4% were at level 5 (Humphrey, 2000). As of March 2000, based on more recent

assessments of 870 organizations since 1995, the percentage distribution by level is: level 1, 39.3%; level 2, 36.3%; level 3, 17.7%; level 4, 4.8%; level 5, 1.8% (Humphrey, 2000). The data indicate that the maturity profile of software organizations is improving.

The SEI maturity assessment framework applies to the organizational or project level. At the individual level and team level, Humphrey developed the Personal Software Processs (PSP) and Team Software Processs (TSP) (Humphrey, 1995, 1997, 2000 a,b). The PSP shows software engineers how to plan and track their work, and good and consistent practices that lead to high-quality software. Time management, good software engineering practices, data tracking, and analysis at the individual level are among the focus areas of PSP. The TSP is built on the PSP and addresses how to apply similar engineering discipline to the full range of a team's software tasks. The PSP and TSP can be viewed as the individual and team versions of the capability maturity model (CMM), respectively. Per Humphrey's guidelines, PSP introduction should follow organizational process improvement and should generally be deferred until the organization is working on achieving at least CMM level 2 (Humphrey, 1995).

Since the early 1990s, a number of capability maturity models have been developed for different disciplines. The Capability Maturity Model Integrationsm (CMMIsm) was developed by integrating practices from four CMMs: for software engineering, for systems engineering, for integrated product and process development (IPPD), and for acquisition. It was released in late 2001 (Software Engineering Institute, 2001a, 2001b). Organizations that want to pursue process improvement across disciplines can now rely on a consistent model. The CMMI has two representations, the staged representation and the continuous representation. The staged representation of the CMMI provides five levels of process maturity.

Maturity Level 1: Initial

Processes are ad hoc and chaotic.

Maturity Level 2: Managed

Addison Wesley: Metrics and Models in Software Quality Engineering, Second Edition2.8 Process Maturity Framework and Quality Standards

Focuses on basic project management. The process areas (PAs) are:

Requirements management Project planning

Project monitoring and control Supplier agreement management Measurement and analysis

Process and product quality assurance Configuration management

Maturity Level 3: Defined

Focuses on process standardization. The process areas are:

Requirements development Technical solution

Product integration Verification

Validation

Organizational process focus Organizational process definition Integrated product management Risk management

Decision analysis and resolution

Organizational environment for integration (IPPD) Integrated teaming (IPPD)