All processes have at least three, and often four, characteristics in common. In addition to producing a specific product or service, they require expenditures of resources and time to produce the product or service; they are expected to deliver the product on time; and from time to time they will produce products with defects or imperfections. Because of these similarities, there are certain fundamental issues associated with processes that everyone concerned with process management shares. These are
• product quality (specifications, tolerances, action limits, and defects) • process time or duration
• product delivery • process cost
These issues tie very closely to the process performance attributes that an organization will want to select for control or improvement. The issues—important to most organizations and common to all software processes—are discussed briefly below. They are readily measured, and understanding the issues provides motivation for all participants in the process.
• Product quality (specifications, tolerances, action limits, and
defects).4 Excessive variability and off-target processes cause defects. Defects introduced into a product have multiple effects. They require the effort of skilled personnel to detect, remove, repair, and retest. Defects also increase process cost, time, and complexity. In addition, defects that escape detection and repair before the product is released to a customer reduce user satisfaction with the product. This is just as true for products that go to internal customers as it is for those that go to external customers. Activities that decrease the introduction of defects or increase the early detection of defects are prime targets for measuring the effectiveness of the process.
• Process duration. Process duration is an issue that translates directly to a process attribute. It is the elapsed time from the start of processing until the product is available to the user. The user in this case is not necessarily the customer or “end user.” It may be a person or team involved in subsequent process activities. The flow of work through a given process can often be improved by providing better tools, using better technology, making more effective use of time, eliminating unnecessary steps, and improving training. • Product delivery. Delivery is a process event. At delivery, the timeliness and quantity of a product or service can be measured against expectations
4Defects occur when products do not meet specifications for designated attributes of product quality.
Tolerances are the amount of leeway granted by specifications. Action limits are limits used to control variability in specified attributes of product quality.
or commitments. If a product or service is not on time (late or early), this may have consequences for the customer or user of the product or service. Delivery is directly related to process duration (via lead time and the time required to execute the process) and delivery frequency.
• Process cost. Process cost includes the cost of executing and supporting the process as well as the costs of understanding and managing the factors that drive costs. As in any other activity, the cost of a software process has many elements and is affected by a variety of factors. Examples include the nature of the product, the statement of work, skills and experience of personnel, labor rates, cost of facilities, overhead, and the pacing of the process. The key to managing process cost is to identify the elements of cost that you can control or affect and look for opportunities to reduce or eliminate the costs. Because of the labor-intensive nature of producing and supporting software, effort is frequently the predominant cost driver for software processes. Measuring how and where people spend their time and relating this to causal factors is important in understanding and controlling the cost of a process [Perry 94]. This often leads to recognizing the need for better tools, training, and different skill mixes.
3.2
Selecting and Defining Measures
…there is nothing more important for transaction of business than use of operational definitions.
W. Edwards Deming, 1986
We now turn to the second planning activity, selecting and defining measures, as highlighted in Figure 3-6. Select and Define Measures Integrate with Software Process Identify Process Issues Goals Objectives
Figure 3-6: Measurement Planning Activities—Step 2
Selecting and defining measures are two different but closely related activities. As we will see, it is one thing to select a measure such as the number of reported defects that we might use to characterize a process or product. It is quite another to create an operational definition for a measure once it has been selected.5 For example, an operational definition
5Operational definitions tell people how measurements are made, and they do this in sufficient detail
for the number of defects found might include the type of defect, when it occurs, when it is detected, the finding activity, whether the defect is static or dynamic, its criticality, and a description of the processes used to find, classify, and count the occurrences of the defect. To have an operational definition, you must know enough about what the data represent and how they are collected (1) to ensure that different people will implement the measures correctly and consistently and (2) to interpret the data correctly.
The activities of selecting and defining measures are coupled in practice because it is not unusual to have to reconsider a measure due to your organization’s inability to apply it easily or consistently once it has been operationally defined. In the pages that follow, we point out several factors to consider when selecting measures. Then we discuss some important characteristics of operational definitions and suggest methods for making your definitions operational.
As you select and define your measures, keep in mind that you will be seeking to apply them to help you control and improve your processes. The primary vehicles that will help you do this are statistical process control and quality improvement methodology. Nonmanufacturing applications of SPC and quality improvement often require ingenuity beyond that normally required for the manufacturing applications where the methods have demonstrated proven value. There seem to be two main reasons why this is so [Montgomery 96]:
1. Most nonmanufacturing operations do not have natural measurement
systems that allow us to define quality easily.
2. The systems that are to be improved are usually fairly obvious in a manufacturing setting, while the observability of processes in a nonmanufacturing setting may be fairly low.
The key to process management and improvement in nonmanufacturing environments is to focus your initial efforts on resolving these two issues.