Assess the relevant information against each hypothesis by asking, “Is this information highly consistent,

consistent, highly inconsis tent, inconsistent, neutral, or not

Death in the Southwest 101

applicable vis-à-vis the hypoth esis?” (The Te@mACH^ software does not include the “neutral” category.)

Analysts using the basic ACH software will have the option of choosing highly consistent (CC), consistent (C), inconsistent (I), highly inconsistent (II), not applicable (NA), or neutral (N). When using basic ACH or My Matrix with Te@mACH^ tool, it is important that analysts code the evidence line by line, in other words horizontally across the matrix, not hypothesis by hypothesis, or vertically down the matrix. Doing so helps the analyst consider each piece of evidence fully against each hypothesis before moving on to the next piece of evidence. This process keeps the analyst focused on the evidence rather than on proving a pet hypothesis. The “Survey” option in Te@mACH^ randomly generates the cells to be coded, thus avoiding this problem.

When entering and coding the data, the credibility score of all evidence or relevant information is set at a default of medium. Analysts can also choose a credibility score of low or high. The software in the basic ACH tool will calculate a weighted inconsistency score that reflects the analysts’

judgment about credibility of the data.

With Te@mACH^, there is a special “Key Assumptions”

box you can check to record and explain any key assumptions relating to a particular item of relevant information. In this case, one might want to note that for

the item “Some people treated with antibiotics recovered,”

doctors could not prove that patients’ recovery was directly connected to the use of antibiotics. The entry “Fort Wingate munitions storage and demo facility is nearby,” also includes an implicit assumption that biological or chemical weapons are or were being processed at the fort and anyone working there could be exposed to toxic substances.

Step 4: Rate the credibility of each item of relevant information.

Step 5: Refine the matrix by reconsidering the hypotheses.

Does it make sense to combine two hypotheses, add a new hypothesis, or disaggregate an existing one?

If the hypotheses are not mutually exclusive, this will become apparent at this stage in the process if the problem did not already surface during the coding process. Analysts should consider disaggregating hypotheses whenever they find themselves “clarifying” the hypothesis as they code.

The trigger, or indicator, that disaggregation is necessary occurs during the coding process. For example, the hypothesis “Deliberate act by extremists,” should be disaggregated to include one hypothesis for terrorists, who might want to target the general population, and a second hypothesis for white supremacists, who would only want to target Navajos or non-Caucasians.

Sometimes hypotheses can be disaggregated into a family of hypotheses. For example, exposure to a toxic substance could involve either a chemical or a biological substance. It could also involve an herbicide or some previously benign substance. It usually is more efficient to first address the overarching hypothesis. If this hypothesis seems likely, then a second ACH analysis can be created breaking the hypothesis into several mutually exclusive components.

Similarly, if the hate-crime hypothesis emerges as a viable explanation, then serious consideration should be given to adding a terrorism hypothesis or a gang-warfare hypothesis.

Step 6: Draw tentative conclusions about the relative likelihood of each hypothesis. An inconsistency score will be calculated by the software; the hypothesis with the lowest inconsistency score is tentatively the most likely hypothesis.

The one with the most inconsistencies is the least likely. The hypotheses with the lowest inconsistency scores appear on the left of the matrix, and those with the highest inconsis-tency scores appear on the right.

It is important to address the likelihood of every hypothesis, not simply the most and least likely. Based upon the above hypotheses and relevant information, some Figure 9.3 ▸ Death in the Southwest ACH Evidence List

tentative conclusions about the relative likelihood of each hypothesis would include the following observations:

▸The “Common Flu” hypothesis is likely to have the most Inconsistents and is the easiest to dismiss.

▸The “Hate Crime” hypothesis also has several Inconsistents and is not likely to be correct.

▸The remaining two hypotheses have the fewest Inconsistents and appear worthy of serious consideration and further investigation.

It is just as important to critically examine the Inconsistent items of relevant information for the most likely hypotheses as well. If many Inconsistents are associated with all the most likely hypotheses, this could signal that there is a missing hypothesis. However, if the inconsistent evidence can be described at best as a “squishy” Inconsistent, then the hypothesis probably is the most likely explanation.

Step 7: Analyze the sensitivity of your tentative conclusion to a change in the interpretation of a few critical items of information, as shown in Figure 9.4. If using the basic ACH software, sort the evidence by diagnosticity, and the most diag nostic information will appear at the top of the matrix.

The Te@mACH^ software will automatically display the most diagnostic information at the top of the matrix.

All of the hypotheses will include at least some inconsistent data. The goal of this step is to understand which pieces of relevant information have the most overall effect on the relative likelihood of the hypotheses and what could happen if those pieces of evidence change.

Step 8: Report the conclusions by considering the relative likelihood of all the hypotheses.

The sensitivity analysis reveals areas for further investigation, but in the absence of additional information, the tentative conclusions about the relative likelihood of the hypotheses hold. However, any written analysis should

Figure 9.4 ▸ Death in the Southwest ACH Sorted by Diagnosticity

Death in the Southwest 103 include a full accounting of conflicting information, gaps,

and assumptions upon which the analysis is based and what new information might change the likelihood of the hypotheses.

Step 9: Identify indicators or milestones for future observation.

The ACH process suggests that analysts should pay careful attention to new information that either corroborates or discredits the two lead hypotheses: New Pathogen or Toxic Substance. Critical questions for further investigation for the New Pathogen hypothesis include the following:

▸What pathogens best match the symptoms that are being reported?

▸Why do Navajos seem particularly susceptible to this new pathogen? What has changed in their environment to make them more susceptible or more exposed to a new pathogen?

▸Do some rodents pose a particular threat? Are some known to carry a pathogen that could produce these symptoms? Are these rodents indigenous to areas populated by Navajos?

Critical questions for further investigation of the Toxic Substance hypothesis include the following:

▸Have any new herbicides been introduced recently by farmers in the Four Corners area?

▸Are there any toxic sites on the lands of the Navajo Nation that could be the cause of the problem?

▸Did any of the victims work at Fort Wingate? Are there toxic dump sites at the fort, or are biological and/or chemical weapons being manufactured or stored there?

Analytic Value Added: As a result of your analysis, what are the most and least likely hypotheses? The two most likely hypotheses are that the people living in the Four Corners area were struck down by a new pathogen or recently exposed to a toxic substance.

What are the most diagnostic pieces of information?

The most diagnostic items of information were the nega-tive tests for flu, the specific symptoms of abdominal/back pain and low blood platelet counts, the lack of reporting of

anti-Navajo rhetoric on the Internet, and the failure of care providers to come down with the same illness.

What, if any, assumptions underlie the data? At the start of the investigation, the CDC investigators were work-ing from two key assumptions: that the cause of the sickness and deaths was either an unknown pathogen or a bioterror-ist act. A corollary to the second assumption was that resi-dents had been exposed to an unannounced or undetected biochemical spill at nearby Fort Wingate.

Are there any gaps in the relevant infor mation that could affect your confidence? Many gaps remain in the evi-dence, as surfaced in the Starbursting and Key Assumptions exercises.

How confident are you in your assessment of the most likely hypotheses? We can be fairly certain that the cause of deaths was not the common flu and moderately confident that Navajos were not deliberately targeted for attack by terrorists or domestic extremists. More research is needed, however, before we can be confident that the cause of death was the introduction of a new pathogen or a recent, sudden exposure to a lethal chemical toxin.

CONCLUSION: THE ANSWER FROM ATLANTA