9 Challenge Analysis
A CAUTIONARY NOTE
9.4 HIGH IMPACT/LOW PROBABILITY ANALYSIS
H
igh Impact/Low Probability Analysis provides decision makers with early warning that a seemingly unlikely event with major policy and resource repercussions might actually occur.When to Use It
High Impact/Low Probability Analysis should be used when one wants to alert decision makers to the possibility that a seemingly long-shot development that would have a major policy or resource impact may be more likely than previously anticipated. Events that would have merited such treatment before they occurred include the reunification of Germany in 1989 and the collapse of the Soviet Union in 1991. This technique might now be used in discussing the potential impact of an outbreak of H5N1 (avian influenza) or applied to a terrorist attack when intent is well-established but there are multiple variations on how and when an attack might be carried out. With regard to the risk of an avian
influenza attack, some in the Homeland Security community maintain that this technique is better described as High Impact/Uncertain Probability Analysis.
A High Impact/Low Probability study usually is initiated when some new and often fragmentary information is received suggesting that a previously unanticipated event might actually occur. For example, it is conceivable that such a tip-off could be received suggesting the need to alert decision makers to the susceptibility of the United States to a major information warfare attack or a dramatic terrorist attack on a national holiday. The technique can also be used to sensitize analysts and
decision makers to the possible effects of low-probability events and stimulate them to think early on about measures that could be taken to deal with the danger or to exploit the opportunity.
“A thoughtful senior policy official has opined that most potentially devastating threats to U.S. interests start out being evaluated as unlikely. The key to effective intelligence-policy relations in strategic warning is for analysts to help policy officials in determining which seemingly unlikely threats are worthy of serious consideration.”
—Jack Davis, “Improving CIA Analytic Performance:
Strategic Warning,” Sherman Kent School for Intelligence Analysis, September 2002.
Value Added
The High Impact/Low Probability Analysis format allows analysts to explore the consequences of an event—particularly one not deemed likely by conventional wisdom—without having to challenge the mainline judgment or to argue with others about how likely an event is to happen. In other words, this technique provides a tactful way of communicating a viewpoint that some recipients might prefer not to hear.
The analytic focus is not on whether something will happen but to take it as a given that an event could happen that would have a major and unanticipated impact. The objective is to explore whether an increasingly credible case can be made for an unlikely event occurring that could pose a major danger—or offer great opportunities. The more nuanced and concrete the analyst’s depiction of the plausible paths to danger, the easier it is for a decision maker to develop a package of policies to protect or advance vital U.S. interests.
Potential Pitfalls
Analysts need to be careful when communicating the likelihood of unlikely events. The meaning of the word “unlikely” can be interpreted as meaning anywhere from 1 percent to 25 percent probability, while “highly unlikely” may mean from 1 percent to 10 percent.12 Customers receiving an intelligence report that uses words of estimative probability such as “very unlikely” will typically interpret the report as consistent with their own prior thinking, if at all possible. If the report says a terrorist attack against a specific U.S. embassy abroad within the next year is very unlikely, it is quite possible, for example, that the analyst may be thinking of about a 10 percent possibility, while a decision maker sees that as consistent with his or her own thinking that the likelihood is less than 1 percent. Such a difference in likelihood can make the difference between a decision to pay or not to pay for expensive contingency planning or a proactive preventive countermeasure. When an analyst is describing the likelihood of an unlikely event, it is desirable to express the likelihood in numeric terms, either as a range (such as less than 5 percent or 10 to 20 percent) or as bettor’s odds (such as 1 chance in 10).
Figure 9.4 shows an example of an unlikely event—the outbreak of conflict in the Arctic Ocean
—that could have major geopolitical consequences for the United States and other neighboring countries. Analysts can employ the technique to sensitize decision makers to the possible effects of the melting of Arctic ice and stimulate them to think about measures that could be taken to deal with the danger.
The Method
An effective High Impact/Low Probability Analysis involves these steps:
Clearly describe the unlikely event.
Define the high-impact consequences if this event occurs. Consider both the actual event and the secondary impacts of the event.
Identify any recent information or reporting suggesting that the likelihood of the unlikely event occurring may be increasing.
Postulate additional triggers that would propel events in this unlikely direction or factors that would greatly accelerate timetables, such as a botched government response, the rise of an energetic challenger, a major terrorist attack, or a surprise electoral outcome that benefits U.S.
interests.
Develop one or more plausible pathways that would explain how this seemingly unlikely event could unfold. Focus on the specifics of what must happen at each stage of the process for the train of events to play out.
Generate a list of indicators that would help analysts and decision makers recognize that events were beginning to unfold in this way.
Identify factors that would deflect a bad outcome or encourage a positive outcome.
Once the list of indicators has been developed, the analyst must periodically review the list.
Such periodic reviews help analysts overcome prevailing mental models that the events being considered are too unlikely to merit serious attention.
Relationship to Other Techniques
High Impact/Low Probability Analysis is sometimes confused with What If? Analysis. Both deal with low-probability or unlikely events. High Impact/Low Probability Analysis is primarily a vehicle for warning decision makers that recent, unanticipated developments suggest that an event previously deemed highly unlikely might actually occur. Based on recent evidence or information, it projects forward to discuss what could occur and the consequences if the event does occur. It challenges the conventional wisdom. What If? Analysis does not require new or anomalous information to serve as a trigger. It reframes the question by assuming that a surprise event has happened. It then looks
backwards from that surprise event to map several ways it could have come about. It also tries to identify actions which, if taken in a timely manner, might have prevented it.
Figure 9.4 High Impact/Low Probability Scenario: Conflict in the Arctic
Source: This example was developed by Michael Bannister and Ray Converse, Pherson Associates, LLC.
Origins of This Technique
The description here is based on Randy Pherson, “High Impact/Low Probability Analysis,” in Handbook of Analytic Tools and Techniques (Reston, Va.: Pherson Associates, LLC, 2008); and Sherman Kent School for Intelligence Analysis training materials.