Identifying Process Issues

All too often our perception is wrong and the answer to our question was OK, but we simply asked the wrong question.

Jim Barr, 1996 (in an internet posting to comp.software-eng)

Process management responsibilities can encompass the entire life cycle of a software product. They can also focus on specific aspects of the development or support cycle. In either case, the process or subprocess that is being managed has a purpose or objective—a raison d’etre that can be traced back to the software organization’s business goals.

Experience has shown that it is important to identify the critical factors that determine whether or not your processes succeed in meeting the goals you set. These critical factors often arise from concerns, problems, or issues that represent levels of risk that threaten your ability to meet your goals, responsibilities, or commitments. They also arise from specifications that must be met to satisfy customer requirements or downstream process needs. We refer to critical factors collectively as issues. Note that issues are not necessarily problems. Rather, based on your understanding and experience with the processes and the products the processes produce, they describe situations that require attention.

Steps for Identifying Process Issues

The following steps outline a straightforward approach for identifying process issues.

1. Clarify your business goals or objectives. You will need to understand how your business goals or objectives relate to your software processes.

In most cases, business goals and objectives that are tied to cost, quality, or time can be mapped readily to the appropriate software processes.

2. Identify the critical processes. Processes that have experienced problems in the past, are executed across organizational boundaries, use a technology for the first time, or operate under higher work loads than have been used before are all prime candidates for your list of critical processes. Processes that provide inputs or support to downstream processes are candidates as well. The list you identify may vary with the

passage of time or with your progress through a development or support cycle.

3. List the objectives for each critical process. Listing objectives in terms of process performance will help you identify potential problem areas. The best way to do this is in terms of the attributes of the products or processes that you want to control or improve. You may find that successfully meeting the objectives for a process depends on the performance of upstream processes. If these processes have not been selected for measurement, you may want to consider adding them to the list created in Step 2.

4. List the potential problem areas associated with the processes. These are concerns or issues that could jeopardize attaining the objectives.

Listing potential problem areas requires that you have a good understanding of your processes and how they relate to each other.

Process flow diagrams and listings of the entities associated with them can help significantly here.

5. Group the list of potential problems into common areas or topics.

This will help you to identify issues that can be described and quantified by closely related measurements.

The Role of Mental Models

One technique for identifying key process issues is to ask questions about the performance of each process and the quality of the products it produces. Your ability to identify critical factors and issues depends on the picture you have in your mind of the tasks, resources, and products required to meet your goals. This picture is often called a mental model [Senge 94]. Sketching or diagramming your mental models for the processes you manage can be very helpful in communicating the issues and measurement possibilities to others, as well as in solidifying your own understanding of the processes. An explicit model of the process under study will visually summarize the relationships among the process tasks. This often brings to light implicit factors that are otherwise overlooked and that may need to be understood more clearly.

Figure 3-3 illustrates the general shape of many process models. Generic models like this are easily elaborated and adapted to specific situations. To construct a specific process model, you should look for

• things the process receives (inputs and resources)—these are used or supplied

• things the process produces (outputs)—these include products, by-products, and effects

• things the process consists of (activities, flow paths, and agents)—the structure of the process

receives produces

Figure 3-3: A Generic Process Model

• things that are expended or consumed (consum-ables)

• things the process holds or retains (internal artifacts, such as inventory and work in progress)

Each “thing” in the process is an entity, and each entity has attributes that characterize some aspect of the process or its products.

Thus, every attribute in a model of a process is a candidate for measurement.

We will return to this generic model and discuss it in more detail later in this chapter.

Figure 3-4 shows a simple model for a

1

Figure 3-4: A Simple Process Model for Defect Tracking

problem management process. The lines from one box to the next indicate the flow of problem reports. The boxes designate the major tasks that must be carried out to move each problem report from the “open” state to the “closed” state. Simple models like Figure 3-4 are useful not only for identifying process issues that bear watching, but also for selecting process attributes for measurement. They are also useful for ensuring that everyone working on process improvement is addressing the same process.

Once you have sketched flowcharts or other pictures that describe a process, step back a bit and examine the process as a whole to see if you have missed anything. By asking questions such as the following, you may discover additional entities whose properties could be worth measuring.

– What determines product quality?

– What determines success?

– What do our customers want?

– What could go wrong?

– What’s not working well?

– What might signal early warnings?

– Where are the delays?

– How big is our backlog?

– Where is backlog occurring?

– What things can we control?

– What limits our capability?

– What limits our performance?

As your knowledge of a process increases, or when the process is inherently large and complex, you may find more sophisticated methods useful. For example, structured control-flow diagrams like the one illustrated in Figure 3-5 can help you to identify important elements of a process and establish mutually understood frameworks for team-based efforts [Henry 92].

Tier 1: Integration Test and Evaluation

Integration Test

Source Code Test Report

Modules Design Requirements

Tier 2: Integration and Evauation Tasks

Determine

Tier 3: Integration Test and Evaluation Procedures

Figure 3-5: Example of a Control-Flow Diagram

Common Process Issues

All processes have at least three, and often four, characteristics in common. In addition to producing a specific product or service, they require expenditures of resources and time to produce the product or service; they are expected to deliver the product on time; and from time to time they will produce products with defects or imperfections. Because of these similarities, there are certain fundamental issues associated with processes that everyone concerned with process management shares. These are

• product quality (specifications, tolerances, action limits, and defects)

• process time or duration

• product delivery

• process cost

These issues tie very closely to the process performance attributes that an organization will want to select for control or improvement. The issues—important to most organizations and common to all software processes—are discussed briefly below. They are readily measured, and understanding the issues provides motivation for all participants in the process.

• Product quality (specifications, tolerances, action limits, and defects).⁴ Excessive variability and off-target processes cause defects.

Defects introduced into a product have multiple effects. They require the effort of skilled personnel to detect, remove, repair, and retest. Defects also increase process cost, time, and complexity. In addition, defects that escape detection and repair before the product is released to a customer reduce user satisfaction with the product. This is just as true for products that go to internal customers as it is for those that go to external customers.

Activities that decrease the introduction of defects or increase the early detection of defects are prime targets for measuring the effectiveness of the process.

• Process duration. Process duration is an issue that translates directly to a process attribute. It is the elapsed time from the start of processing until the product is available to the user. The user in this case is not necessarily the customer or “end user.” It may be a person or team involved in subsequent process activities. The flow of work through a given process can often be improved by providing better tools, using better technology, making more effective use of time, eliminating unnecessary steps, and improving training.

• Product delivery. Delivery is a process event. At delivery, the timeliness and quantity of a product or service can be measured against expectations

4Defects occur when products do not meet specifications for designated attributes of product quality.

Tolerances are the amount of leeway granted by specifications. Action limits are limits used to control variability in specified attributes of product quality.

or commitments. If a product or service is not on time (late or early), this may have consequences for the customer or user of the product or service.

Delivery is directly related to process duration (via lead time and the time required to execute the process) and delivery frequency.

• Process cost. Process cost includes the cost of executing and supporting the process as well as the costs of understanding and managing the factors that drive costs. As in any other activity, the cost of a software process has many elements and is affected by a variety of factors. Examples include the nature of the product, the statement of work, skills and experience of personnel, labor rates, cost of facilities, overhead, and the pacing of the process. The key to managing process cost is to identify the elements of cost that you can control or affect and look for opportunities to reduce or eliminate the costs. Because of the labor-intensive nature of producing and supporting software, effort is frequently the predominant cost driver for software processes. Measuring how and where people spend their time and relating this to causal factors is important in understanding and controlling the cost of a process [Perry 94]. This often leads to recognizing the need for better tools, training, and different skill mixes.