Returning to our hypothetical experiment on cell- phone use, let’s suppose that participants’ driving performance was impaired while they talked on a cell phone. If our experiment was conducted properly, it will have high internal validity and thus we can be confident that talking on the phone, and not some other factor, caused the driv- ing impairment. There remain, however, other questions that we must ask. Would the results be similar with other types of participants or when driving under different road or traffic conditions? These questions focus on external validity, which is the degree to which the results of a study can be generalized to other populations, settings, and
conditions. Judgments about external validity typically do not focus on the exact responses of the participants. For example, in any particular exper- iment the fact that talking on a cell phone might increase drivers’ braking reaction time by exactly 114 milliseconds is not the issue. Rather, we are concerned about the external validity of the general underlying principle: Does talking on the phone while driving impair drivers’ performance?
To determine external validity, either we or other scientists need to replicate our experiment.
Replication is the process of repeating a study to deter-
mine whether the original findings can be duplicated. If our findings are successfully replicated, especially when studying other types of participants and driving conditions, we become more confident in concluding that cell-phone use impairs driving performance. Indeed, in simulation experiments talking on a cell phone while driving has been found to interfere with driving performance in ru- ral environments and urban environments of vary- ing complexity, among younger and older drivers, and when using hand-held and hands-free phones (Strayer & Drews, 2004; Törnros & Bolling, 2006).
Increasingly, psychologists are paying more attention to cross-cultural replication. For example, German Posada and his coworkers (2002) studied interactions between mothers and their infant girls and boys. Middle-class families from Denver (United States) and Bogotá (Colombia) partici- pated. In both samples, infants showed a closer
“IT WAS MORE OF A ‘TRIPLE-BLIND’ TEST. THE PATIENTS DIDN'T KNOW WHICH ONES WERE GETTING THE REAL DRUG, THE DOCTORS DIDN'T KNOW, AND I'M AFRAID, NOBODY KNEW”
FIGURE 2.18
The double-blind procedure is useful, but scientists try to avoid the infamous “triple-blind procedure.” Copyright © 2000 by Sidney Harris: ScienceCartoonsPlus.com.
Reprinted with permission.
Focus 18
What is external validity? Why is replication important? Apply these concepts to paranormal claims.
emotional bond to mothers who were more sensi- tive and responsive to their baby’s needs. Because this study was correlational, it does not demon- strate cause-effect relations. Still, replicating the findings across two cultural groups within the same study increases our confidence in the gener- alizability of the association between sensitive caregiving and infant attachment.
When research findings fail to replicate, it may lead to better research and new discoveries as sci- entists search for clues to explain why the results turned out differently in one study versus another. For example, although many experiments suggest
that cell-phone use interferes with optimal driving performance, not all experiments do. Further re- search will be needed to sort out the factors, such as different driving conditions, that might account for such results.
Studies that consistently fail to replicate the re- sults of earlier research also may suggest that the original research may have been flawed or that the finding was a fluke. Even so, the scientific process has done its job and prevented us from getting caught in a blind alley. To see why replication is such an important component of the scientific process, let’s look at the following “Beneath the Surface.”
Beneath the Surface
Science, Psychics, and the Paranormal
Do you believe—or know people who believe—in psychic phenomena, such as mental telepathy (transmitting thoughts between individuals) and precognition (foretelling the future; Figure 2.19)? Surveys around the world reveal widespread public belief in the paranormal (Alcock, 2003). Adopting a scientific attitude means we should approach this issue with open-minded skepticism; that is, we should apply rigorous standards of evaluation, as we do to all phenomena (Cardeña et al., 2000). The ability of independent investigators to repli- cate initial research findings is one of those standards.
When tested under controlled conditions in well- designed experiments and replications, claim after claim of psychic ability has evaporated. In 1976, the Committee for the Scientific Investigation of Claims of the Paranormal was formed. It consists of psychologists, other scientists, philoso- phers, and magicians expert in the art of fakery. To conclude
that a phenomenon is psychic, the committee requires that presently known natural physical or psychological explana- tions be ruled out. To date, it has not judged any psychic claims to be valid.
What about paranormal demonstrations by self- proclaimed psychics, such as using mental powers to bend spoons? About 30 years ago, James Randi, a magician and expert in the art of psychic fraud, began offering $10,000 to anyone who could demonstrate paranormal ability under his scrutiny. Today the offer is $1 million, and still no one has collected. Predictions made by leading psychics in national newspapers also yield dismal results (Emery Jr., 2001).
In the 1990s, a report in a major scientific journal pro- vided evidence of mental telepathy from 11 studies using the ganzfeld procedure (Bem & Honorton, 1994). In this approach, a participant (the “receiver”) listens to a hissing sound played through earphones and sees red light through translu- cent goggles. Parapsychologists believe this procedure makes the receiver more sensitive to mental telepathy signals. In another shielded room, the “sender” concentrates on one of four different visual forms presented in random order over a series of trials. In these studies, the receivers reported the correct form on 32 percent of the trials, a statistically signifi- cant increase above the chance level of 25 percent.
Does the ganzfeld procedure—which involves many rigorous controls—provide the first solid evidence of a psy- chic phenomenon? Some scientists suggest that the original ganzfeld studies may not have fully prevented the receivers from detecting extremely subtle cues that could have influ- enced their responses (Hyman, 1994). Although several para- psychology researchers have reported successful replications (Parker, 2000), psychologists Julie Milton and Richard Wise- man (1999) analyzed 30 ganzfeld studies conducted by seven independent laboratories and concluded that “the ganzfeld technique does not at present offer a replicable method for producing ESP in the laboratory” (p. 387, italics added). As
Continued
FIGURE 2.19