STATISTICAL PROCESS CONTROL

SOFTWARE ACQUISITION G

G

G

O

O

O

L

L

L

D

D

D

P

P

P

R

R

R

A

A

A

C

C

C

T

T

T

I

I

I

C

C

C

E

E

E

STATISTICAL PROCESS CONTROL

FOCUS AREA: QUALITY - Measurement

DESCRIPTION OF THE PRACTICE:

SUMMARY DESCRIPTION

Statistical Process Control (SPC) can be applied to software development processes. A process has one or more outputs, as depicted in the first figure below. These outputs, in turn, have measurable attributes. SPC is based on the idea that these attributes have two sources of variation: natural (also known as common) and assignable (also known as special) causes. If the observed variability of the attributes of a process is within the range of variability from natural causes, the process is said to be under statistical control. The practitioner of SPC tracks the variability of the process to be controlled. When that variability exceeds the range to be expected from natural causes, one then identifies and corrects assignable causes.

Output

Process

Measure

Attribute

Identify

and Correct

Assignable Cause

The key steps for implementing Statistical Process Control are:

o Identify defined processes

o Identify measurable attributes of the process

o Characterize natural variation of attributes

o Track process variation

o If the process is in control, continue to track

o If the process is not in control: - Identify assignable cause - Remove assignable cause

DETAILED DESCRIPTION

Statistical Process Control (SPC) can be applied to software development processes. A process has one or more outputs, as depicted in Figure 1. These outputs, in turn, have measurable attributes. SPC is based on the idea that these attributes have two sources of variation: natural (also known as common) and assignable (also known as special) causes. If the observed variability of the attributes of a process is within the range of variability from natural causes, the process is said to be under statistical control. The practitioner of SPC tracks the variability of the process to be controlled. When that variability exceeds the range to be expected from natural causes, one then identifies and corrects assignable causes. Figure 2 depicts the steps in an implementation of SPC.

Output

Process

Measure

Attribute

Identify

and Correct

Assignable Cause

Characterize Natural Variation of Attributes

Yes

Track Variation

No

Identify Defined

Processes

Figure 2: How To Perform SPC

In practice, reports of SPC in software development and maintenance tend to concentrate on a few software processes. Specifically, SPC has been used to control software (formal)

inspections, testing, maintenance, and personal process improvement. Control charts are the most common tools for determining whether a software process is under statistical control. A variety of types of control charts are used in SPC. Table 1, based on a survey [Radice 2000] of SPC usage in organizations attaining Level 4 or higher on the SEI CMM metric of process maturity, shows what types are most commonly used in applying SPC to software. The

combination of an Upper Control Limit (UCL) and a Lower Control Limit (LCL) specify, on control charts, the variability due to natural causes. Table 2 shows the levels commonly used in setting control limits for software SPC. Table 3 shows the most common statistical techniques, other than control charts, used in software SPC. Some of these techniques are used in trial

applications of SPC to explore the natural variability of processes. Some are used in techniques for eliminating assignable causes. Analysis of defects is the most common technique for

eliminating assignable causes. Causal Analysis-related techniques, such as Pareto analysis, Ishikawa diagrams, the Nominal Group Technique (NGT), and brainstorming, are also frequently used for eliminating assignable causes.

Table 1: Usage of Control Charts

Type of Control/Attribute Chart Percentage

Xbar-mR 33.3%

u-Chart 23.3%

Xbar 13.3%

c-Chart 6.7%

z-Chart 6.7%

Not clearly stated 16.7%

From Ron Radice’s survey of 25 CMM Level 4 and Level 5 organizations [Radice 2000]

Table 2: Location of UCL-LCL in Control Charts

Location Percentage Three-sigma 16% Two-sigma 4% One-Sigma 8% Combination 16% None/Not Clear 24%

From Ron Radice’s survey of 25 CMM level 4 and level 5 organizations [Radice 2000]

Table 3: Usage of Other Statistical Techniques

Statistical Technique Percentage Run Charts 22.8% Histograms 21.1% Pareto Analysis 21.1% Scatter Diagrams 10.5% Regression Analysis 7.0% Pie Charts 3.5% Radar/Kiviat Charts 3.5% Other 10.5%

From Ron Radice’s survey of 25 CMM level 4 and level 5 organizations [Radice 2000]

Control charts are a central technology for SPC. Figure 3 shows a sample control chart

constructed from simulated data. This is an X-chart, where the value of the attribute is graphed. Control limits are graphed. In this case, the control limits are based on a priori knowledge of the distribution of the attribute when the process is under control. The control limits are at three sigma. For a normal distribution, 0.2% of samples would fall outside the limits by chance. This

control chart indicates the process is out of control. If this control chart were for real data, the next step would be to investigate the process to identify assignable causes and to correct them, thereby bringing the process under control.

Lower Control Limit

Upper Control Limit

Attribute

Sample

Figure 3: A Control Chart

Some have extended the focus of SPC in applying it to software processes. In manufacturing, the primary focus of control charts is to bring the process back into control. In software, the product is also a focus. When a software process exceeds the control limits, rework is typically performed on the product. In manufacturing, the cost of stopping a process is high. In software, the cost of stopping is lower, and few shutdown and startup activities are needed [Jalote and Saxena 2002].

SPC is one way of applying statistics to software engineering. Other opportunities for applying statistics exist in software engineering. Table 4 shows, by lifecycle phase, some of these uses of statistics. The National Research Council recently sponsored the Panel on Statistical Methods in Software Engineering [NRC 1996]. The panel recommended a wide range of areas for applying statistics, from visualizing test and metric data to conducting controlled experiments to

demonstrate new methodologies.

Table 4: Some Applications of Statistics in Software Engineering

Phase Use of Statistics

Requirements Specify performance goals that can be measured statistically, e.g., no more than 50 total field faults and zero critical faults with 90% confidence.

Design Pareto analysis to identify fault-prone modules. Use of design of experiments in making design decisions empirically.

Coding Statistical control charts applied to inspections.

Testing Coverage metrics provides attributes. Design of experiments useful in

creating test suites. Statistical usage testing is based on specified operational profile. Reliability models can be applied.

Those applying SPC to industrial organizations, in general, have built process improvements on top of SPC. The focus of SPC is on removing variation caused by assignable causes. As defined here, SPC is not intended to lower process variation resulting from natural causes. Many corporations, however, have extended their SPC efforts with Six Sigma programs. Six Sigma provides continuous process improvement and attempts to reduce the natural variation in processes. Typically, Six Sigma programs use the “Seven Tools of Quality” (Table 5). The Shewhart Cycle (Figure 4) is a fundamental idea for continuous process improvement.

Table 5: The Seven Tools of Quality

Tool Example of Use

Check Sheet To count occurrences of problems.

Histogram To identify central tendencies and any skewing to one side or the other.

Pareto Chart To identify the 20% of the modules which yield 80% of the issues.

Cause and Effect Diagram

For identifying assignable causes.

Scatter Diagram For identifying correlation and suggesting causation.

Control Chart For identifying processes that are out of control.

Graph For visually displaying data, e.g., in a pie chart.

Act; apply the improvement to the process throughout or abandon it if improvements not found. Study/Check; Do the collected data show an

improvement or increase your understanding of the

process? Plan to improve a process, collect data, and create a schedule.

Do; Implement the improvement on a

small scale.

PDSA

Cycle for

Continuous

Improvement

CHARACTERISTICS OF IMPLEMENTATION:

SUMMARY CHARACTERISTICS

NO DATA CURRENTLY AVAILABLE

ANTICIPATED BENEFITS OF IMPLEMENTATION:

SPC is a powerful tool to optimize the amount of information needed for use in making management decisions [Eickelmann and Anant 2003]. Statistical techniques provide an understanding of the business baselines, insights for process improvements, communication of value and results of processes, and active and visible involvement. SPC provides real time analysis to establish controllable process baselines; learn, set, and dynamically improve process capabilities; and focus business on areas needing improvement. SPC moves away from opinion-based decision making [Radice 2000].

These benefits of SPC cannot be obtained immediately by all organizations. SPC requires defined processes and a discipline of following them. It requires a climate in which personnel are not punished when problems are detected. It requires management commitment [Demmy 1989].

DETAILED CHARACTERISTICS

The processes controllable by SPC are unlimited in application domain, lifecycle phase, and design methodology. Processes need to exhibit certain characteristics to be suitable for SPC (as shown in the table below). In addition, a process to which SPC is applied should be

homogeneous. For example, applications of SPC to software inspections have found that inspections must often be decomposed to apply SPC effectively. Florence [1999] found, for instance, that the inspection of human machine interface specifications should be treated as a process separate from the inspection of application specifications in the project he examined.

Weller [2000] found that inspections of new and revised code should be treated as separate processes in the project he examined. A trial application of SPC is useful in identifying homogeneous sub processes. Issues other than identifying defined homogeneous processes arise in implementing SPC. A second table presents such implementation issues.

Criteria Of Processes Suitable for SPC

• Well-defined

• Have attributes with observable measures

• Repetitive

• Sufficiently critical to justify monitoring effort

(Based on [Demmy 1989])

SPC Implementation Issues

Define Process Consistent measurements cannot be expected from software processes that are not documented and generally followed.

Choose Appropriate Measures

Measures need not be exhaustive. One or two measures that provide insight into the performance of a process or activity are adequate, especially if the measures are related to the process or activity goal. Measures that can be tracked inexpensively are preferable.

Focus on Process Trends

Control charts should be constructed so as to detect process trends, not individual nonconforming events.

Calculate Control Limits Correctly

Straightforward formulas exist for calculating control limits and analyzing distributions. Introductory college courses in statistics usually do not address process-control techniques in detail.

Investigate and Act

SPC only signals the possible existence of a problem. Without detailed investigations, as in an audit, and instituting corrective action, SPC will not provide any benefit.

Provide Training

Problems in following the above recommendations for implementing SPC can be decreased with effective training. SPC training based on examples of software processes is to be preferred.

RELATIONSHIPS TO OTHER PRACTICES:

The Figure below represents a high-level process architecture for the subject practice, depicting relationships among this practice and the nature of the influences on the practice (describing how other practices might relate to this practice). These relationship statements are based on

definitions of specific “best practices” found in the literature and the notion that the successful implementation of practices may “influence” (or are influenced by) the ability to successfully

Process Architecture for the "Statistical Process Control" Gold PracticeTM implement other practices. A brief description of these influences is included in the table below.

Summary of Relationship Factors

Statistical

Process

Control (SPC)

Provide data upon which decisions can be based Determine which attributes, at what levels, should be controlled Communicate progress towards controlling processes Assess progress towards process control Improve testing efficiency and effectiveness

INPUTS TO THE PRACTICE

Determine which attributes, at only be effective if the most critical

what levels, should be controlled

Statistical Process Control can

processes are identified and addressed using the technique. Practices that help establish clear goals and decision points, and are based on meaningful metrics and attributes based on specific program or technical goals, stand to gain the most payback from using SPC. SPC techniques need not be restricted to the present, i.e., planning for the insertion of new technology later in the life cycle should also plan for the use of SPC to ensure that processes are controlled and reliability of the resulting software artifacts is optimized.

Define whether environment is

eration

appropriate for process control

Practices such as Performance-Based Specifications and Commercial Specifications/Open System can imply the gen and collection of data. Such data may serve as appropriate input to SPC techniques such as control charts. Therefore, an

environment that is data-rich provides an excellent opportunity to leverage the benefit statistical process control techniques.

Provide data upon which decisions can be based

An initial step in applying SPC is often to discover controllable and homogeneous processes. Past performance data can be used for this purpose. Formal inspection processes and processes for leveraging COTS/NDI explicitly call for metrics to be collected. These metrics can be used as the basis for SPC.

Summary of Relationship Factors (continued) OUTPUTS FROM THE PRACTICE Assess progress towards

process control

SPC can be used not only to control processes, but also to determine if quantitative requirements on software processes are being met. The results of SPC, then, provide valuable data and information that can be used to manage progress towards achievement of software requirements. Part of this ability to manage progress is supported by the quantitative progress measurements that are inherent in the SPC process, primarily in the form of defect tracking and correction against specific, quantitative quality targets.

Communicate progress towards controlling processes

Management go/no-go decisions can be based on whether development processes are under control. SPC presents graphical displays supporting such decisions. The number and types of available graphical formats that can be used as part of the SPC process provide accessible visibility into progress being made to all program stakeholders. Demonstration-based reviews provide an excellent vehicle for communicating the progress being made in controlling processes through the use of SPC.

Improve Testing Efficiency and Effectiveness

By providing control over software development processes, SPC will result in more predictable and more reliable software.

Rigorous testing that is guided by specifications and supported by well-documented and accurate operational and usage-based models will be much more effective under the controlled processes resulting from SPC.

RESOURCES:

Websites

0 http://www.asq.org/ American Society for Quality

0 http://www.asq-software.org/ Software Division of the American Society for Quality 0 http://www.sixsigmaforum.com/ Six Sigma Forum established by the American Society

for Quality (ASQ)

0 http://www.isixsigma.com/ A Web portal devoted to Six Sigma programs. 0 http://www.qualitydigest.com/ Quality Digest. A magazine.

0 http://www2.umassd.edu/SWPI/ Software Process Resource Collection

0 http://www.psmsc.com/ Practical Software and Systems Measurement. Sponsored by the U.S. Department of Defense and the U.S. Army.

0 http://deming.eng.clemson.edu/ A page at Clemson University for Continuous Quality Improvement, including software and tutorials.

0 http://www.statsoftinc.com/textbook/stathome.html An online textbook in statistics 0 http://www.math.uah.edu/stat/ Virtual Laboratory in Probability and Statistics (Kyle

Siegrist’s tutorial) Tools and Methods

Tools for plotting control charts, plotting other graphs, and calculating various statistics support SPC. Common spreadsheets, such as Excel, provide much of the needed capabilities, but a number of vendors have produced tools customized for SPC. Some are listed below:

• http://www.unido.org/en/doc/4268 Measurement Control Chart Toolkit (MCCT). A toolkit for SPC available from the United Nations Industrial Development Organization.

• http://deming.eng.clemson.edu/ A page at Clemson University for Continuous Quality Improvement, including software and tutorials.

• http://www.qualityamerica.com/ A company providing SPC software, training, and consulting

• http://www.qualitran.com/SPC_Software/body_spc_software.html A windows-based SPC and Process Improvement software tool sold by Qualitrain Professional Services Incorporated.

• http://www.spc-software-package.com/ A software package provided by the company To The Point.

• http://www.cse.iitk.ac.in/users/jalote/SDA.html Pankaj Jalote’s Software Process Design and Analysis page. Includes an applet for computing control limits in control charts.

Experts/Contact Points

0 http://www.reifer.com/ Donald Reifer, a consultant. 0 http://www.rspa.com/ R. S. Pressman and Associates

0 http://www.sqe.com/ Software Quality Engineering. A private company providing training and consulting.

0 http://www.qualityamerica.com/ A company providing SPC software, training, and consulting.

Training Opportunities:

• http://www.sei.cmu.edu/products/courses/spc-sw.html Software Engineering Institute (SEI) course on SPC.

• http://www.qualityamerica.com/ A company providing SPC software, training, and consulting

• http://www.sqe.com/ Software Quality Engineering. A private company providing training and consulting.

Bibliography: [Bertolino, et. al.

2002] A. Bertolino, E. Marchetti, R. Mirandola, G. Lombardi, and E. Peciola, _{“Experience of Applying Statistical Control Techniques to the Function Test} Phase of a Large Telecommunications System,” IEE Proceedings –

Software, V. 149, N. 4, August 2002, pp. 93-101

[Card 1994] _{D. Card, “Statistical Process Control for Software?” IEEE Software, V. 11,} No. 3, May 1994, pp. 95-97

[Cobb and Mills

1990] R. H. Cobb and H. D. Mills, “Engineering Software under Statistical Quality _{Control”, IEEE Software, V. 7, No. 6, November 1990, pp. 44-54} [Dalal, et. al. 1993] _{S. R. Dalal, J. R. Horgan, J. R. Kettenring, “Reliable Software and}

Communication: Software Quality, Reliability, and Safety”, Proceedings of

the 15th International Conference on Software Engineering, 17-21 May 1993

[De Lucia, et. al.

2002] A. De Lucia, A. Pannella, E. Pompella, S. Stefanucci, “Empirical Analysis of _{Massive Maintenance Processes,” Proceedings of the Sixth European}

Conference on Software Maintenance and Reengineering (CSMR’02), 2002

[Demmy 1989] _{W. S. Demmy, “Statistical Process Control in Software Quality Assurance”,}

Proceedings of the IEEE National Aerospace and Electronics Conference

(NAECON 1989), 22-26 May 1989, pp. 1585-1590

[Florac and

Carleton 1999] W. A. Florac and A. D. Carleton, _{Process Control for Software Process Improvement}Measuring the Software Process: Statistical _{. Addison-Wesley, 1999} [Florac, et. al. 2000] _{W. Florac, A. D. Carleton, and J. R. Barnard, “Statistical Process Control:}

Analyzing a Space Shuttle Onboard Software Process”, IEEE Software, V. 17, no. 4, July/August 2000, pp. 97-106

[Florence 1999] _{A. Florence, “SEI CMM Level 4 Quantitative Analysis”, Twenty-Fourth Annual} Software Engineering Workshop

[French 1995] _{V. A. French, “Applying Software Engineering and Process Improvement to} Legacy Defence System Maintenance: An Experience Report,” Proceedings

of the International Conference on Software Maintenance, 12-20 October

1995 [Gardiner and

Montgomery 1987] J. S. Gardiner and D. C. Montgomery, “Using Statistical Control Charts for Software Quality Control,” Quality and Reliability Engineering International, V. 3, 1987, pp. 40-43

[Hayes 1998] _{W. Hayes, “Using a Personal Software Process to Improve Performance,”}

Proceedings of the Fifth International Software Metrics Symposium, 20-21

November 1998

[Ishikawa 1982] _{K. Ishikawa, Guide to Quality Control, Unipub/Quality Resources, 1982} [Jacob and Pillai

[Jalote and Saxena

2002] P. Jalote and A. Saxena, “Optimum Control Limits for Employing Statistical _{Process Control in Software Process”, IEEE Transactions on Software}

Engineering, V. 28, N. 12, December 2002, pp. 1126-1134

[NRC 1996] _{Panel on Statistical Methods in Software Engineering of the National} Research Council, Statistical Software Engineering, National Academy Press, 1996

[Paulk, et. al. 1995] _{M. C. Paulk et. al., The Capability Maturity Model: Guidelines for Improving}

the Software Process, Addison-Wesley, 1995.

[Prowell, et. al.

1999] S. J. Prowell, C. J. Trammell, R. C. Linger, and J. H. Poore, _{Software Engineering: Technology and Process. Addison-Wesley, 1999} Cleanroom [Radice 2000] _{R. Radice, “Statistical Process Control in Level 4 and 5 Organizations}

Worldwide”, Proceedings of the 12th Annual Software Technology

Conference, 2000. (Also at http://www.stt.com/)

[Turner 2002] _{R. Turner, Implementation of Best Practices in US Department of Defense}

Software-Intensive Systems, Dissertation, George Washington University,

January 2002.

[Weller 2000] _{E. F. Weller, “Practical Applications of Statistical Process Control”, IEEE}

APPENDICES

DEFINITIONS:

An application of statistics to controlling industrial processes, including processes in software development and maintenance. Statistical Process Control (SPC) is used to identify and remove variations in processes that exceed the variation to be expected from natural causes.

The purpose of process control is to detect any abnormality in the process.

SOURCES (Origins of the Practice):

Walter Shewhart developed Statistical Process Control (SPC) in the 1920s. Shewhart sought methods applying statistics to industrial practice. Acceptance testing and SPC grew out of this work. Shewhart proposed the use of control charts, a core technique for SPC, in a historic internal memorandum of 16 May 1924 at Bell Telephone Laboratories.

For a long time, SPC was most widely adopted in Japan, not the United States. Shewhart mentored W. Edwards Deming, and Deming went on to introduce quality technologies into Japanese industry. The ”Guide to Quality Control” Ishikawa [1982], first published in 1968 in Japanese, is a guide to quality control techniques that became prevalent in Japan after World War II. Corporations in the United States began adopting quality technology, including SPC, more widely in the 1980s. Recently, many United States corporations have instituted Six Sigma programs. These programs, through continual process improvement, attempt to reduce the natural variation in industrial processes.

Some began applying SPC techniques to software in the 1980s. Gardiner and Montgomery [1987] report an example. Software inspections seem to provide the processes that are most commonly monitored with SPC in software. Some recently proposed software process models include opportunities for SPC. The spiral lifecycle provides a natural time for tuning software processes, namely before the start of the development of each increment. SPC yields analyzed data that managers can use in selecting processes to tune. Cleanroom software engineering combines incremental development and software inspections with other technologies, such as reliability modeling. The Software Engineering Institute (SEI) Capability Maturity Model (CMM) mandates that SPC be used in Level 4 organizations.

RECOMMENDING SOURCES:

•

SPC is described as part of the Key Process Area, Quantitative Process Management, in the CMM. This Key Process Area must be implemented to achieve Level 4, Managed, in the CMM.

“The purpose of Quantitative Process Management is to control the process performance of the software process quantitatively. Software process performance represents the actual results achieved from following a software process.

Quantitative Process Management involves establishing goals for the performance of the project’s defined software process, which is described in the Integrated Software

Management key process area, taking measurements of the process performance, analyzing these measurements, and making adjustments to maintain process

performance within acceptable limits. When the process is stabilized within acceptable limits, the project’s defined software process, the associated measurements, and the acceptable limits for the measurements are established as a baseline and used to control process performance quantitatively.” [Paulk, et. al. 1995]

This process area is renamed Organizational Process Performance at Maturity Level 4 in the CMMI, Version 1.1.

•

Turner surveyed a sample of 35 software experts from academia, industry, and government participating in various software-intensive working groups. The survey partitioned 32 best practices into four focus areas. Statistical Process Control, a best practice identified by Donald Reifer, was grouped into the Measurement focus area. The best practices were ranked based on the 23 survey respondents’ individual rankings; rating of effectiveness; and groupings into the eight best practices, eight worst practices, or otherwise. SPC was ranked thirtieth out of the 32 best practices.

•

Cleanroom alludes to a technique used in semiconductor fabrication to prevent defects, and is a methodology for software development that integrates several software engineering technologies: 0 Incremental development 0 Formal specifications 0 Stepwise refinement 0 Structured programming 0 Formal verifications 0 Formal reviews 0 Statistical testing

0 Certification (reliability) measurement

Incremental development supports SPC:

Incremental development is based on the engineering principle of controlled iteration in product development. Rather than a single pass through the development process, incremental development involves a series of smaller, cumulative development passes. Each pass (increment) is cumulative, involving all work in previous increments plus some new work…

In addition to the benefits of intellectual control, customer feedback, and risk

management, incremental development enables the project team to employ statistical process control. Product quality is measured at the end of each increment and is compared with the team’s quality goals. The deviation between actual results and goals is used to determine whether the project is under control. A minor deviation confirms that the project is on track, whereas an unacceptable deviation occasions a careful

performance review. If problems are identified, the team can make process changes to improve performance in the next increment. [Prowell, et. al., 1999]

GLOSSARY

c-Chart A control chart displaying counts per item where the range of the counts is fixed. Contrast with a u-chart.

CE Diagram Cause and Effect diagram. A diagram, developed by Kaoru Ishikawa, that resembles a fish skeleton. The diagram shows the causes and sub causes leading to an effect.

CMM Capability Maturity Model

Control Chart A graph with limit lines that is used to detect changes in the process from the graphed data is collected. [Ishikawa 1982]

Fishbone Diagram

See CE Diagram.

Histogram A graph showing for defining intervals, the number of values in a sample or the percentage in that interval. A visual display of a probability distribution.

Ishikawa Diagram

See CE Diagram.

Kiviat Chart A chart in the shape of a circle with evenly spaced radii, where each radius represents the value of a (non-negative attribute). Attributes are often plotted with respect to user-defined thresholds.

LCL Lower Control Limit. Used in constructing a control chart.

NGT Nominal Group Technique

Pareto Analysis The use of a bar chart that displays by frequency, in descending order, the most important defects. Proper use of this chart will have the cumulative percentage on a second y-axis (to the right of the chart). If the Pareto principle is evident, about 20% of the categories on the far left will have about 80% of the impact on the problem.

p Chart A control chart in which a fraction formed from the ratio of discrete variables (e.g., fraction of items defective) is plotted against time.

Pie Chart A graph in which the percentage in one of a small number of categories is displayed as an area in a circle.

Radar Chart See Kiviat Chart

Regression Analysis

A statistical technique for determining the best mathematical expression describing the functional relationship between one response and one or more independent variables. (From

http://www.sixsigmaforum.com/terms/r.html)

Run-Chart A performance measure of a process over a specified period of time used to identify trends or patterns.

Scatter Diagram

A plot in which two attributes of the data are plotted, one on the abscissa and the other on the ordinate. Useful for detecting a relationship between the two attributes.

SEI Software Engineering Institute

u-Chart A control chart displaying counts per item (e.g., total defects per thousand source lines of code) where the range of the counts is not fixed. Contrast with the c-chart.

UCL Upper Control Limit. Used in constructing a control chart.

X-Chart Individual control chart

Xbar Chart A control chart in which the average for subgroups of data is plotted by subgroup.

Xbar-mR Chart A control chart in which both the average and the range for subgroups of data are plotted by subgroup.

XmR Chart Individual and Moving Range control chart.

z-Chart A control chart in which the plotted data has been transformed to be from a normal distribution if the process is under control.

CASE STUDIES FROM THE LITERATURE:

The table below summarizes recent experiences of a number of organizations with SPC. Examples of the Use of SPC

SPC Technique

Application Process/Attributes Discussion/Results Reference

Control chart Military software unit maintaining almost 380,000 source lines of code

Defects per Person-Day and Defects per Thousand Lines of Code.

SPC techniques introduced along with software

engineering project management techniques, inspections, and metrics. Process improvement resulted in 20% more code being produced than on previous version, rework in Integration and Test dropping from 4.7 to 0.09 person-years. [French 1995] X Control Chart Process automation and consumer electronics projects in C++, ranging from 150 to 400 Function Points Defects per Thousand Lines of Code for code review and testing. Lines of Code per Hour for inspection

preparation.

Illustrates how using control charts identified out-of-control processes. Corrective actions were implemented to remove assignable causes. [Jacob and Pillai 2003] XmR Control Chart GCOS 8 (an operating system)

Inspection review rate and defect-detection effectiveness of software inspections.

Identified sub processes (inspections of new code and revised code). Demonstrated inspections were under control. Estimated defects remaining in the code.

[Weller 2000] XmR Control Chart and u-Chart Space Shuttle Onboard Software Inspection review rate, defect-detected in inspections and test. Trial study of SPC. Identified inspection sub processes and natural variation of these processes. [Florac, et. al. 2000] Z-Chart A sample of 298 engineers training in the Software Engineering Institute’s Personal Software Process (PSP) Defects discovered in design reviews.

Process control charts presented in the context of metrics collected by engineers going through PSP training.

Demonstrations of the effectiveness of the PSP based on size estimation accuracy, effort estimation accuracy, defects per source line remove early in

[Hayes 1998]

the lifecycle, and productivity. Confidence intervals based on statistical models of failures Ericsson Lab, Italy Effectiveness in failure detection of function testing.

Pilot study. Concluded process control for function test can be based on examined models. [Bertolino, et. al. 2002] Regression model, Outlier analysis, Box plot Large Y2K maintenance project. Part of a major international software enterprise in Caserta, Italy.

Size and productivity metrics during maintenance.

Study analyzed correlation, characterized patterns of variations, developed regression model to predict impact of maintenance. Concluded that

maintenance process is predictable, repeatable, and suitable for SPC.

[De Lucia, et. al. 2002]