An analogy takes into consideration that no new program, no matter how state of the art it may be technologically, represents a totally new system. Most new programs evolve from programs already fielded that have had new features added on or that simply represent a new combination of existing components. The analogy method uses this concept for estimating new components, subsystems, or total programs. That is, an analogy uses actual costs from a similar program with adjustments to account for differences 39 Expert opinion, also known as engineering judgment, is commonly applied to fill gaps in a relatively detailed WBS when one or
between the requirements of the existing and new systems. A cost estimator typically uses this method early in a program’s life cycle, when insufficient actual cost data are available but the technical and program definition is good enough to make the necessary adjustments.
Adjustments should be made as objectively as possible, by using factors (sometimes scaling parameters) that represent differences in size, performance, technology, or complexity. The cost estimator should identify the important cost drivers, determine how the old item relates to the new item, and decide how each cost driver affects the overall cost. All estimates based on the analogy method, however, must pass the “reasonable person” test—that is, the sources of the analogy and any adjustments must be logical, credible, and acceptable to a reasonable person. In addition, since analogies are one-to-one comparisons, the historical and new systems should have a strong parallel.
Analogy relies a great deal on expert opinion to modify the existing system data to approximate the new system. If possible, the adjustments should be quantitative rather than qualitative, avoiding subjective judgments as much as possible. An analogy is often used as a cross-check for other methods. Even when an analyst is using a more detailed cost estimating technique, an analogy can provide a useful sanity check. Table 12 shows how an analogy works.
Table 12: An Example of the Analogy Cost Estimating Method
Parameter Existing system New system
Cost of new system
(assuming a linear relationship)
Engine F-100 F-200 Thrust 12,000 lbs 16,000 lbs
Cost $5.2 million X (16,000/12,000) x $5.2 million = $6.9 million Source: © 2003, Society of Cost Estimating and Analysis (SCEA), “Costing Techniques.”
The equation in table 12 implicitly assumes a linear relationship between engine cost and amount of thrust. However, there should be a compelling scientific or engineering reason why an engine’s cost is directly proportional to its thrust. Without more data (or an expert on engine costs), it is hard to know what parameters are the true drivers of cost. Therefore, when using the analogy method, it is important that the estimator research and discuss with program experts the reasonableness of technical program drivers to determine whether they are significant cost drivers.
The analogy method has several advantages:
It can be used before detailed program requirements are known. ■
If the analogy is strong, the estimate will be defensible. ■
An analogy can be developed quickly and at minimum cost. ■
The tie to historical data is simple enough to be readily understood. ■
Analogies also have some disadvantages: An analogy relies on a single data point. ■
It is often difficult to find the detailed cost, technical, and program data required for analogies. ■
There is a tendency to be too subjective about the technical parameter adjustment factors. ■
The last disadvantage can be best explained with an example. If a cost estimator assumes that a new component will be 20 percent more complex but cannot explain why, this adjustment factor is unacceptable. The complexity must be related to the system’s parameters, such as that the new system will have 20 percent more data processing capacity or will weigh 20 percent more. Case study 34 highlights what can happen when technical parameter assumptions are too optimistic.
Case Study 34: Cost Estimating Methods, from Space Acquisitions,
GAO-07-96
In 2004, Advanced Extremely High Frequency (AEHF) satellite program decision makers relied on the program office cost estimate rather than the independent estimate the CAIG developed to support the production decision. The program office estimated that the system would cost about $6 billion, on the assumption that AEHF would have 10 times more capacity than Milstar, the predecessor satellite, at half the cost and weight. However, the CAIG concluded that the program could not deliver more data capacity at half the weight, given the state of the technology. In fact, the CAIG believed that to get the desired increase in data rate, the weight would have to increase proportionally. As a result, the CAIG estimated that AEHF would cost $8.7 billion and predicted a $2.7 billion cost overrun. The CAIG relied on weight data from historical satellites to estimate the program’s
cost, because it considered weight to be the best cost predictor for military satellite communications. The historical data from the AEHF contractor showed that the weight had more than doubled since the program began and that the majority of the weight growth was in the payload. The Air Force also used weight as a cost predictor but attributed the weight growth to structural components rather than the more costly payload portion of the satellite. The CAIG stated that major cost growth was inevitable from the program start because historical data showed that it was possible to achieve a weight reduction or an increase in data capacity but not both at the same time.
GAO, Space Acquisitions: DOD Needs to Take More Action to Address Unrealistic Initial Cost Estimates of Space Systems, GAO-07-96 (Washington, D.C.: Nov. 17, 2006).