6 Scenarios and Indicators
6.1 SCENARIOS ANALYSIS
6.1.3 The Method: Multiple Scenarios Generation
Multiple Scenarios Generation is similar to Alternative Futures Analysis except that with this
technique, you are not limited to two critical drivers generating four scenarios. By using multiple 2 × 2 matrices pairing every possible combination of multiple driving forces, you can create a very large number of possible scenarios. This is sometimes desirable to make sure nothing has been overlooked.
Once generated, the scenarios can be screened quickly without detailed analysis of each one. Once sensitized to these different scenarios, analysts are more likely to pay attention to outlying data that would suggest that events are playing out in a way not previously imagined.
Training and an experienced facilitator are needed to use this technique. Here are the basic steps:
Clearly define the focal issue and the specific goals of the futures exercise.
Brainstorm to identify the key forces, factors, or events that are most likely to influence how the issue will develop over a specified time period.
Define the two ends of the spectrum for each driver.
Pair the drivers in a series of 2 × 2 matrices.
Develop a story or two for each quadrant of each 2 × 2 matrix.
From all the scenarios generated, select those most deserving of attention because they illustrate compelling and challenging futures not yet being considered.
Develop indicators for each scenario that could be tracked to determine whether or not the scenario is developing.
The technique can be illustrated by exploring the focal question “What is the future of the insurgency in Iraq?” (See Figure 6.1.3a.) Here are the steps:
Convene a group of experts (including some creative thinkers who can challenge the group’s
mental model) to brainstorm the forces and factors that are likely to determine the future of the insurgency in Iraq.
Select from this list those factors or drivers whose outcome is the hardest to predict or for which analysts cannot confidently assess how the driver will influence future events. In the Iraq example, three drivers meet these criteria:
The role of neighboring states (Iran, Syria)
The capability of Iraq’s security services (police and military) The political environment in Iraq
Figure 6.1.3a Multiple Scenarios Generation: Future of the Iraq Insurgency
Define the ends of the spectrum for each driver. For example, the neighboring state could be stable and supportive at one end and unstable and disruptive at the other end of the spectrum.
Pair the drivers in a series of 2 × 2 matrices as shown in Figure 6.1.3a.
Develop a story or a couple of stories describing how events might unfold for each quadrant of each 2 × 2 matrix. For example, in the 2 × 2 matrix that is defined by the role of neighboring states and the capability of Iraq’s security forces, analysts would be tasked with describing how the insurgency would function in each quadrant on the basis of the criteria defined at the far end of each spectrum. In the upper left hand quadrant, the criteria would be stable and supportive neighboring states but ineffective internal security capabilities. (See Figure 6.1.3b.) In this
“world,” one might imagine a regional defense umbrella that would help to secure the borders.
Another possibility is that the neighboring states would have the Shiites and Kurds under control, with the only remaining insurgents Sunnis, who continue to harass the Shia-led central government.
Review all the stories generated, and select those most deserving of attention. For example, which scenario
Presents the greatest challenges to Iraqi and U.S. decision makers?
Raises particular concerns that have not been anticipated?
Surfaces new dynamics that should be addressed?
Suggests new collection needs?
Figure 6.1.3b Future of the Iraq Insurgency: Using Spectrums to Define Potential Outcomes
Select a few scenarios that might be described as “wild cards” (low probability/high impact developments) or “nightmare scenarios” (see Figure 6.1.3c).
Consider what decision makers might do to prevent bad scenarios from occurring or enable good scenarios to develop.
Generate a list of key indicators to help monitor which scenario story best describes how events are beginning to play out.
Relationship to Other Techniques
Any scenario analysis might be followed by constructing a Cross-Impact Matrix to identify and
analyze potential interactions or feedback loops between the various driving forces in each scenario.
Multiple Scenarios Generation is a specific application of Morphological Analysis, described in chapter 5.
Figure 6.1.3c Selecting Attention-deserving and Nightmare Scenarios
Origins of This Technique
Scenarios Analysis is a broad concept that can be implemented in various ways for a variety of
purposes. Three variations of Scenarios Analysis that seem most useful for intelligence analysis were selected for description here. The model of Simple Scenarios was developed by Pherson Associates, LLC. Alternative Futures Analysis and Multiple Scenarios Generation were previously described in Randy Pherson, Handbook of Analytic Tools and Techniques (Reston, Va.: Pherson Associates, LLC, 2008). For information on other approaches to Scenarios Analysis, see Andy Hines, “The Current State of Scenario Development: An Overview of Techniques,” Foresight 9, no. 1 (March 2007). The Multiple Scenarios Generation illustrations are drawn from a report prepared by Alan Schwartz (PolicyFutures, LLC), “Scenarios for the Insurgency in Iraq,” Special Report 174 (Washington, D.C.:
United States Institute of Peace, October 2006).
6.2 INDICATORS
I
ndictors are observable phenomena that can be periodically reviewed to help track events, spot emerging trends, and warn of unanticipated changes. An indicators list is a pre-established set of observable or potentially observable actions, conditions, facts, or events whose simultaneous occurrence would argue strongly that a phenomenon is present or is very likely to occur. Indicators can be monitored to obtain tactical, operational, or strategic warnings of some future development that, if it were to occur, would have a major impact.The identification and monitoring of indicators are fundamental tasks of intelligence analysis, as they are the principal means of avoiding surprise. They are often described as forward-looking or predictive indicators. In the law enforcement community indicators are also used to assess whether a target’s activities or behavior is consistent with an established pattern. These are often described as backward-looking or descriptive indicators.
When to Use It
Indicators provide an objective baseline for tracking events, instilling rigor into the analytic process, and enhancing the credibility of the final product. Descriptive indicators are best used to help the analyst assess whether there are sufficient grounds to believe that a specific action is taking place.
They provide a systematic way to validate a hypothesis or help substantiate an emerging viewpoint.
Figure 6.2a is an example of a list of descriptive indicators, in this case pointing to a clandestine drug laboratory.
A classic application of forward-looking indicators is to seek early warning of some
undesirable event, such as a military attack or a nuclear test by a foreign country. Today indicators are often paired with scenarios to identify which of several possible scenarios is developing. They are also used to measure change that points toward an undesirable condition, such as political instability or a humanitarian crisis, or toward a desirable condition, such as economic reform or
democratization. Analysts can use this technique whenever they need to track a specific situation to monitor, detect, or evaluate change over time. In the private sector, indicators are used to track
whether a new business strategy is working or whether a low-probability scenario is developing that offers new commercial opportunities.
Value Added
The human mind sometimes sees what it expects to see and can overlook the unexpected.
Identification of indicators creates an awareness that prepares the mind to recognize early signs of significant change. Change often happens so gradually that analysts don’t see it, or they rationalize it as not being of fundamental importance until it is too obvious to ignore. Once analysts take a position on an issue, they can be reluctant to change their minds in response to new evidence. By specifying in advance the threshold for what actions or events would be significant and might cause them to change their minds, analysts can seek to avoid this type of rationalization.
Figure 6.2a Descriptive Indicators of a Clandestine Drug Laboratory
Source: Pamphlet from ALERT Unit, New Jersey State Police, 1990; republished in The Community Model, Counterdrug Intelligence Coordinating Group, 2003.
Defining explicit criteria for tracking and judging the course of events makes the analytic process more visible and available for scrutiny by others, thus enhancing the credibility of analytic judgments. Including an indicators list in the finished product helps decision makers track future developments and builds a more concrete case for the analytic conclusions.
Preparation of a detailed indicator list by a group of knowledgeable analysts is usually a good learning experience for all participants. It can be a useful medium for an exchange of knowledge between analysts from different organizations or those with different types of expertise—for example, analysts who specialize in a particular country and those who are knowledgeable about a particular field, such as military mobilization, political instability, or economic development.
The indicator list becomes the basis for directing collection efforts and for routing relevant information to all interested parties. It can also serve as the basis for the analyst’s filing system to keep track of these indicators.
When analysts or decision makers are sharply divided over the interpretation of events (for example, how the war in Iraq or Afghanistan is progressing), of the guilt or innocence of a “person of interest,” or the culpability of a counterintelligence suspect, indicators can help depersonalize the debate by shifting attention away from personal viewpoints to more objective criteria. Emotions often can be diffused and substantive disagreements clarified if all parties agree in advance on a set of criteria that would demonstrate that developments are—or are not—moving in a particular direction or that a person’s behavior suggests that he or she is guilty as suspected or is indeed a spy.
Potential Pitfalls
The quality of indicators is critical, as poor indicators lead to analytic failure. For these reasons, analysts must periodically review the validity and relevance of an indicators list. Narrowly conceived or outdated indicators can reinforce analytic bias, encourage analysts to discard new evidence, and lull consumers of information inappropriately. Indicators can also prove to be invalid over time, or they may turn out to be poor “pointers” to what they were supposed to show. By
regularly checking the validity of the indicators, analysts may also discover that their original
assumptions were flawed. Finally, if an opponent learns what indicators are on your list, the opponent may make operational changes to conceal what you are looking for or arrange for you to see contrary indicators.
The Method
The first step in using this technique is to create a list of indicators. (See Figure 6.2b for a sample indicators list.) The second step is to monitor these indicators regularly to detect signs of change.
Developing the indicator list can range from a simple process to a sophisticated team effort. For example, with minimum effort you could jot down a list of things you would expect to see if a particular situation were to develop as feared or foreseen. Or you could join with others to define multiple variables that would influence a situation and then rank the value of each variable based on incoming information about relevant events, activities, or official statements. In both cases, some form of brainstorming, hypothesis generation, or scenario development is often used to identify the indicators.
Figure 6.2b Using Indicators to Track Emerging Scenarios in Zambria
Source: 2009 Pherson Associates, LLC.
A good indicator must meet several criteria, including the following:
Observable and collectible. There must be some reasonable expectation that, if present, the indicator will be observed and reported by a reliable source. If an indicator is to monitor change over time, it must be collectable over time.
Valid. An indicator must be clearly relevant to the end state the analyst is trying to predict or assess, and it must be inconsistent with all or at least some of the alternative explanations or outcomes. It must accurately measure the concept or phenomenon at issue.
Reliable. Data collection must be consistent when comparable methods are used. Those observing and collecting data must observe the same things. Reliability requires precise definition of the indicators.
Stable. An indicator must be useful over time to allow comparisons and to track events. Ideally, the indicator should be observable early in the evolution of a development so that analysts and decision makers have time to react accordingly.
Unique. An indicator should measure only one thing and, in combination with other indicators, should point only to the phenomenon being studied. Valuable indicators are those that are not only consistent with a specified scenario or hypothesis but are also inconsistent with alternative
scenarios or hypotheses. The Indicators Validator tool, described later in this chapter, can be used to check the diagnosticity of indicators.
Any indicator list used to monitor whether something has happened, is happening, or will happen implies at least one alternative scenario or hypothesis—that it has not happened, is not happening, or will not happen. Many indicators that a scenario or hypothesis is happening are just the opposite of indicators that it is not happening, but some are not. Some are consistent with two or more scenarios or hypotheses. Therefore, an analyst should prepare separate lists of indicators for each scenario or hypothesis. For example, consider indicators of an opponent’s preparations for a military attack where there may be three hypotheses—no attack, attack, and feigned intent to attack with the goal of forcing a favorable negotiated solution. Almost all indicators of an imminent attack are also
consistent with the hypothesis of a feigned attack. The analyst must identify indicators capable of diagnosing the difference between true intent to attack and feigned intent to attack. The mobilization of reserves is such a diagnostic indicator. It is so costly that it is not usually undertaken unless there is a strong presumption that the reserves will be needed.
Maintaining separate indicator lists for alternative scenarios or hypotheses is particularly useful when making a case that a certain event is unlikely to happen, as in What If? Analysis or High
Impact/Low Probability Analysis.
After creating the indicator list or lists, you or the analytic team should regularly review incoming reporting and note any changes in the indicators. To the extent possible, you or the team should decide well in advance which critical indicators, if observed, will serve as early-warning decision points. In other words, if a certain indicator or set of indicators is observed, it will trigger a report advising of some modification in the intelligence appraisal of the situation.
Techniques for increasing the sophistication and credibility of an indicator list include the following:
Establishing a scale for rating each indicator Providing specific definitions of each indicator
Rating the indicators on a scheduled basis (e.g., monthly, quarterly, or annually) Assigning a level of confidence to each rating
Providing a narrative description for each point on the rating scale, describing what one would expect to observe at that level
Listing the sources of information used in generating the rating
Figure 6.2c on the next page is an example of a complex indicators chart that incorporates the first three techniques listed above.
Relationship to Other Techniques
Indicators are closely related to a number of other techniques. Some form of brainstorming is
commonly used to draw upon the expertise in creating indicators of multiple analysts with different perspectives and different specialties. The development of alternative scenarios should always involve the development and monitoring of indicators that point toward which scenario is evolving.
What If? Analysis and High Impact/Low Probability Analysis depend upon the development and use of indicators. Indicators are often entered as items of evidence in Analysis of Competing Hypotheses.
The Indicators Validator, which is discussed in the next section, is a tool used to test the diagnosticity of indicators.
Figure 6.2c Zambria Political Instability Indicators
Origins of This Technique
The identification and monitoring of indicators of military attack is one of the oldest forms of
intelligence analysis. The discussion here is based on Randy Pherson, “Indicators,” in Handbook of Analytic Tools and Techniques (Reston, Va.: Pherson Associates, LLC, 2008); Pherson, The
Indicators Handbook (Reston, Va.: Pherson Associates, LLC, 2008); and the CIA’s Sherman Kent School for Intelligence Analysis training materials. Cynthia M. Grabo’s book, Anticipating Surprise:
Analysis for Strategic Warning (Lanham, Md.: University Press of America, 2004), is a classic text on the development and use of Indicators.