EXPERIMENTATION FOR ROOT CAUSE ANALYSIS

Some hypotheses can be supported or rejected by a simple observation to see if they fit facts. Others may require experimentation to evaluate them.

A treatment is the set of conditions in an experiment. During an experi-ment, the treatment variable is a factor that is manipulated or adjusted by the experimenter to determine its effect on the response variable, which is the outcome of the experiment. Confounding variables or noise are vari-ables that can affect the results of an experiment but are uncontrolled.

Variables should be controlled whenever possible or otherwise dealt with when not controllable. One method of dealing with confounding variables is blocking. An uncontrolled variable is distributed across all treatments by ensuring all experimental groups or blocks contain the confounding variable, thereby decreasing variability and increasing the precision of the experimental results. Box advises to “block what you can and randomize what you cannot” (Box, Hunter, and Hunter, 2005, p. 93).

Randomization can cancel or minimize the effects of noise and thereby increase the accuracy of experimental results. To randomize, samples should be randomly selected, such as taking them from multiple levels in a box and not from the top; it is possible that something changed during the production process and the parts on the top of the box are not representa-tive of the entire population. Replication is also used to increase the accu-racy as well as the precision of an experiment. Repeating an experiment can provide better data by including variation in the results. A treatment

The Science of Root Cause Analysis • 65 that is not replicated may be influenced by variation and does not provide as accurate a picture of the true results as multiple experimental runs.

The precision of the results is their closeness to each other across mul-tiple measurements or experiments. Accuracy is the closeness of experi-mental or measurement results to the true value. The ideal is both precision and accuracy. An experiment could produce results with high preci-sion and low accuracy; the results would be consistent, but wrong.

An experimenter should also use operational definitions, which are quantitative descriptions of terms that are used so that the meanings of the terms are unambiguous. Operational definitions must be clear enough for anybody using them to understand and should use measurements or tests to define the terms (Deming, 1989).

It may be advantageous to establish a baseline, the output of a process prior to changes being made. An experiment may indicate that changes to the treatment variable resulted in a change to the response variable;

however, it is also possible that the response variable would have changed because of an unknown noise factor regardless of the setting of the treat-ment variable. The experitreat-mental results should be compared to the baseline.

Key Points

• A treatment, also known as an experimental run, is the set of condi-tions during an experiment.

• A factor is a condition that affects an output, for example, tempera-ture, material type, mixtempera-ture, settings on a machine, or pressure.

• The treatment variable, also known as an independent variable, is a factor that is manipulated by the experimenter to determine its effect or lack of effect on the response variable.

• A response variable, also known as the dependent variable, is the result of the manipulation of the treatment variable.

• The confounding variable, also known as the confounding factor, is a source of noise.

• Noise in an experiment is an uncontrolled and potentially unknown factor that influences the experimental results.

• Precision is the closeness of measurements to each other.

• Accuracy is the closeness of a measurement to the true value.

• Blocking reduces variability and increases precision by spreading confounding variables across the experimental results.

• Randomization increases the accuracy of experimental results by canceling out the influence of noise.

• Replication is the repetition of an experiment to increase the accu-racy and precision of the results.

• Operational definitions are clear quantitative descriptions of terms using tests or measurements to define the terms.

• Failing to check the baseline may result in attributing changes in the response variable to the setting of the treatment variable when no actual relationship exists and the response variable would have changed regardless of the setting of the treatment variable.

• Blinding may be needed to increase objectivity.

eXAMPLe 9.3

A quality engineer is investigating the root cause of shrinkage porosity in an aluminum die- cast part. The hypothesis is that “shrinkage porosity is the result of insufficient pressure during the casting process.” The die- cast machine pressure was set at 150 kN during the production run that resulted in many parts with porosity, so the experiment will determine if higher pressure eliminates the problem. High and low pressures are not operational definitions, so the quality engineer defines low pressure as 150 kN and high pressure as 195 kN. Porosity is defined operationally as an open space on the surface of a die- cast aluminum part.

The treatment variable is casting pressure, and the response variable is the presence or absence of porosity on the finished sample part. Potential confounding variables are material type, material volume, ejector operat-ing cycle, first- stage velocity, and second- stage velocity. The quality engi-neer records the machine settings for the confounding variables and ensures that they stay the same during each experimental run. The material quantity must also stay the same, and all material used must come from the same source to eliminate variability.

The quality engineer makes a trial run using the previous pressure setting of 150 kN to establish a baseline. If the part produced is without porosity, then the experiment cannot continue without modification; the experimental test condition is expected to produce a part free of porosity.

However, the results would be without value if the initial condition were also without porosity. It would not be appropriate to conclude that higher pressure eliminated the porosity and therefore low pressure was the root cause of the porosity.

The baseline experiment resulted in porosity. The experiment then is run under the experimental high- pressure condition, and there is no poros-ity present. The entire experiment is then repeated five times to ensure the results are consistent.

The Science of Root Cause Analysis • 67

PRoceDURe

Step 1: Create a test plan based on a hypothesis. The predicted result of the hypothesis is the response variable.

Step 2: Determine the treatment conditions by establishing the treatment variable or variables.

Step 3: Identify potential confounding variables and establish a method to eliminate, control, or minimize them. Blocking and randomiza-tion may be useful here.

Step 4: Ensure all terms are written as operational definitions.

Step 5: Establish the baseline if necessary or possible.

Step 6: Perform the experiment.

Step 7: Replicate the experiment and compare the results; a large dif-ference is an indicator that variation is present and more replicates are needed.

69

10