EVALUATING ALTERNATIVE RESEARCH APPROACHES

Having now reviewed the three major research strategies, we are in a position to evaluate them in detail. As we already have noted, each has strengths and limitations (Table 2.2).

Case Studies and Clinical Research: Strengths and Limitations

A major advantage of case studies, particularly as they are conducted in clini- cal settings, is that they overcome the potential superfi ciality and artifi ciality of correlational and experimental methods. In a case study, the investigator

Table 2.2 Summary of Potential Strengths and Limitations of Alternative Research Methods

Potential Strengths Potential Limitations

CASE STUDIES AND CLINICAL RESEARCH

1. Avoid the artifi ciality of laboratory 1. Lead to unsystematic observation

2. Study the full complexity of 2. Encourage subjective interpretation of data person–environment relationships

3. Lead to in-depth study of individuals 3. Do not establish causal relationships QUESTIONNAIRES AND CORRELATION RESEARCH

1. Study a wide range of variables 1. Establish relationships that are associational rather than causal

2. Study relationships among many 2. Problems of reliability and validity of self-report variables questionnaires 3. Large samples easily obtained 3. Individuals not studied in depth

LABORATORY STUDIES AND EXPERIMENTAL RESEARCH

1. Manipulate specifi c variables 1. Exclude phenomena that cannot be studied in the laboratory 2. Record data objectively 2. Create an artifi cial setting that limits the generality of fi ndings 3. Establish cause–effect relationships 3. Foster demand characteristics and experimenter expectancy

effects

c02TheScientificStudyofPeople.indd Page 57 07/11/12 7:19 PM user-019A

learns about deeply important aspects of an individual’s life, which may not occur in a brief experiment or a survey questionnaire. Clinicians conducting case studies directly observe how the client thinks and feels about events. One examines the behavior of interest directly and does not have to extrapolate from a somewhat artifi cial setting to the real world.

A further advantage is that clinical research may be the only feasible way of studying some phenomena. When one needs to study the full complexity of per- sonality processes, individual environment relationships, and the within-person organization of personality, in-depth case studies may be the only option.

In-depth study of a few individuals has two main features that stand in con- trast with research on groups (Pervin, 1983). First, relationships established for a group as a whole may not refl ect the way any individual behaves or the way some subgroups of individuals behave. An average learning curve, for example, may not refl ect the way any one individual learns. Second, by consid- ering only group data, one may miss some valuable insights into processes going on in particular individuals. Some time ago, Henry Murray argued for the use of individual as well as group studies as follows: “In lay words, the subjects who gave the majority response may have done so for different rea- sons. Furthermore, a statistical answer leaves unexplained the uncommon (exhibited-by-the-minority) response. One can only ignore it as an unhappy exception to the rule. Averages obliterate the ‘individual characters of individ- ual organisms’ and so fail to reveal the complex interaction of forces which determine each concrete event” (1938, p. viii).

Regarding limitations of the case study method, two signifi cant drawbacks can be noted: (1) fi ndings from one case study may not generalize to other peo- ple and (2) the case study method cannot demonstrate causality, that is, that one psychological process causally infl uences another. In personality science, as in any science, researchers hope to identify the causes of the phenomena they study. They wish not only to describe a person, but to determine how and why different elements of personality affect one another. A case study may provide a wonderful description, but it generally cannot provide a defi nite causal explana- tion. For example, imagine a clinical case study that describes changes in an individual’s psychological well-being that occur over the course of a year-long clinical treatment. The case study may describe the changes accurately, but it cannot demonstrate that treatment caused the changes being described. Life events other than clinical treatment may have had causal infl uence.

There is a third limitation: Case studies often rely on the subjective impressions of researchers. Unlike the correlational and experimental strate- gies, which commonly employ objective measurement procedures, case stud- ies commonly rest on impressionistic reports, such as a therapist’s subjective impressions of his or her client. Such reports may refl ect not only the psy- chological qualities of the person being studied but the qualities—the beliefs, expectations, and biases—of the psychologist who prepares the report. There is no guarantee that a different researcher examining the same case would come to the same conclusions. This subjective element can lower the reliabil- ity and validity of case-study evidence.

The Use of Verbal Reports Clinical research in personality need not involve the use

of verbal reports by subjects, though clearly it often does. In making use of verbal reports, we are confronted with special problems associated with such

THREE GENERAL STRATEGIES OF RESEARCH ₅₉

data. Treating what people say as accurate refl ections of what has actually oc- curred or is actually going on has come under attack from two very different groups. First, psychoanalysts and dynamically oriented psychologists (Chapters 3 and 4) argue that people often distort things for unconscious reasons: “Chil- dren perceive inaccurately, are very little conscious of their inner states and re- tain fallacious recollections of occurrences. Many adults are hardly better” (Murray, 1938, p. 15). Second, many experimental psychologists argue that peo- ple do not have access to their internal processes and respond to interviewer questions in terms of some inferences they make about what must have been going on rather than accurately reporting what actually occurred (Nisbett & Wilson, 1977; Wilson, Hull, & Johnson, 1981). For example, despite experiment- er evidence that subjects make decisions in accord with certain experimental manipulations, the subjects themselves may report having behaved in a particu- lar way for very different reasons. Or, to take another example, when consumers are asked about why they purchased a product in a supermarket, they may give a reason that is very different from what can experimentally be demonstrated to have been the case. In a sense, people give subjective reasons for behaving as they do but may not give the actual causes. In sum, the argument is that whether for defensive reasons or because of “normal” problems people have in keeping track of their internal processes, verbal self-reports are questionable sources of reliable and valid data (West & Finch, 1997; Wilson, 1994).

Other psychologists argue that verbal reports should be accepted for what they are—data (Ericsson & Simon, 1993). Essentially, the argument states that there is no intrinsic reason to treat verbal reports as any less useful data than an overt motor response, such as pressing a lever. Indeed, it is possible to ana- lyze the verbal responses of people in as objective, systematic, and quantitative a fashion as their other behavioral responses. If verbal responses are not automatically discounted, then the question becomes, Which kinds of verbal responses are most useful and trustworthy? Here the argument is made that subjects can only report about things they are presently attending to or have already handled. If the experimenter asks the subject to remember or explain things that were never attended to in the fi rst place, the subject will either make an inference or state a hypothesis about what occurred (White, 1980). Thus, if you later ask persons why they purchased one product over another in the su- permarket when they were not attending to this decision at the time, they will give you an inference or a hypothesis rather than an account of what occurred. Those who argue in favor of the use of verbal reports suggest that when they are elicited with care and the circumstances involved are appreciated, they can be a useful source of information. Although the term introspection (i.e., verbal descriptions of a process going on inside a person) was discredited long ago by experimental psychologists, there is now increased interest in the potential use of such data. In accepting the potential use of verbal reports, we may expand the universe of potential data for rich and meaningful observation. At the same time, we must keep in mind the goals and requirements of reliability and valid- ity. Thus, we must insist on evidence that the same observations and interpre- tations can be made by other investigators and that the data do refl ect the concepts they are presumed to measure. In appreciating the merits and vast potential of verbal reports, we must also be aware of the potential for misuse and naive interpretation. In sum, verbal reports as data should receive the same scrutiny as other research observations.

c02TheScientificStudyofPeople.indd Page 59 07/11/12 7:19 PM user-019A

Correlational Research and Questionnaires: Strengths and Limitations

A main advantage of correlational studies using questionnaires is sample size. It often is possible to study large numbers of people. By conducting research through the Internet, psychologists can obtain extremely large and diverse samples of participants (Fraley, 2007).

Another advantage of the correlational approach concerns reliability. Many questionnaires provide extremely reliable indices of the psychological con- structs they are designed to measure. This is important because tests must be reliable in order to detect important features of personality that might be over- looked otherwise. For example, researchers fi nd that individual differences in personality traits are highly stable over time; people who differ in extraversion or conscientiousness in young adulthood will probably differ in middle and later adulthood as well (e.g., Costa & McCrae, 2002). One could not detect this fact unless the measures of the personality traits were highly reliable.

Correlational studies have been enormously popular among personality psy- chologists. Yet it is important to be aware of three limitations of this research strategy. The fi rst limitation is one that differentiates correlational studies from case studies. Case studies provide richly detailed information about an individ- ual. In contrast, correlational studies provide relatively superfi cial information about individual persons. A correlational study will provide information about an individual’s scores on the various personality tests that happen to have been used in the research. But if there are some other variables that are important to an individual person, a correlational study generally will not reveal them.

The second limitation is one that case studies and correlational studies share. As in a case study, in a correlational study it is diffi cult to draw fi rm conclusions about causality. The fact that two variables are correlated does not mean that one variable necessarily caused the other. A “third variable” could have infl uenced both of the variables in one’s study and caused those variables to be correlated. For example, in the nun study, it is possible that some psychological, biological, or environmental factor that was not measured in the study caused some nuns to experience fewer positive emotions and to live less long. As a hypothetical example, if one conducted a study akin to the nun study with college students, one might fi nd that positive emotionality would predict longevity. But that would not necessarily mean that the ten- dency to experience positive emotions during college caused people to live longer. For example, levels of academic success could function as a third vari- able. Students who are doing extremely well in college might experience more positive emotions as a result of their academic success. They also might obtain more lucrative jobs after graduation, again as a result of their academic suc- cess. Their high-paying jobs might enable them to pay for superior health care, which in turn could lengthen their life whether or not they continue to experi- ence frequent position emotions. In this hypothetical example, emotions and length of life would be correlated, but not because of any direct causal connec- tion between the two.

A third limitation concerns the widespread reliance on self-report question- naires. When people are describing themselves on a questionnaire, they may be biased to answer items in a way that has nothing to do with the exact content of the items or the psychological construct that the psychologist is trying to as- sess. These biases are called response styles . Two illustrative response-style

THREE GENERAL STRATEGIES OF RESEARCH ₆₁

problems can be considered. The fi rst is called acquiescence . It involves the ten- dency to agree consistently (or disagree consistently) with items regardless of their content. For example, a test-taker may prefer to say “Yes” or “I agree” when asked questions, rather than saying “No” or “I disagree.” The second re- sponse style is called social desirability . Instead of responding to the intended psychological meaning of a test item, a subject may respond to the fact that dif- ferent types of responses are more or less desirable. If, hypothetically, a test item asks “Have you ever stolen anything from a store?,” the answer “No” is clearly a more socially desirable response than “Yes.” If people are biased to answer questions in a socially desirable manner, then their test scores may not accurately refl ect their true psychological characteristics.

A research report that highlights the problem of distortion of questionnaire responses, while also emphasizing the potential value of clinical judgment, is that of Shedler, Mayman, and Manis (1993). In this research conducted by psy- chologists with a psychoanalytic orientation who were skeptical of accepting self-report data at face value, individuals who “looked good” on mental health questionnaire scales were evaluated by a psychodynamically oriented clinician. On the basis of his clinical judgments, two subgroups were distinguished: one defi ned as being genuinely psychologically healthy in agreement with the ques- tionnaire scales and a second defi ned as consisting of individuals who were psychologically distressed but who maintained an illusion of mental health through defensive denial of their diffi culties. Individuals in the two groups were found to differ signifi cantly in their responses to stress. Subjects in the illusory mental health group were found to show much higher levels of coronary reac- tivity to stress than subjects in the genuinely healthy group. Indeed, the former subjects were found to show even greater levels of coronary reactivity to stress than subjects who reported their distress on the mental health questionnaire scales. The differences in reactivity to stress between the genuinely healthy sub- jects and the “illusory” healthy subjects were considered not only to be statisti- cally signifi cant but medically signifi cant as well. Thus, it was concluded that “for some people, mental health scales appear to be legitimate measures of mental health. For other people, these scales appear to measure defensive de- nial. There seems to be no way to know from the test score alone what is being measured in any given respondent” (Shedler et al., 1993, p. 1128).

Those who defend the use of questionnaires note that such problems often can be eliminated through careful test construction and interpretation. Psychologists can reduce or eliminate the effects of acquiescence by varying the wording of items on a test so that consistent “yes” responses do not give one a higher overall test score. They can employ questionnaires that are specifi - cally designed to measure the degree to which a given person tends to endorse socially desirable responses. Comprehensive personality questionnaires com- monly include test items or scales to measure whether subjects are faking or trying to present themselves in a particularly favorable or socially desirable way. Including such scales in a research project, however, often is inconve- nient or costly, and thus, such scales often are lacking in particular studies.

Laboratory, Experimental Research: Strengths and Limitations

In many ways, our ideal image of scientifi c investigation is laboratory research. Ask people for their description of a scientist, and they are likely to conjure up

c02TheScientificStudyofPeople.indd Page 61 07/11/12 7:19 PM user-019A

an image of someone in a sterile lab. As we have already seen, this image is too limited; personality psychologists employ a range of scientifi c methods, and laboratory research is but one of them. Yet it is an important one. The experi- mental approach, as we have noted, has the unique ability to manipulate vari- ables of interest and thereby to establish cause–effect relationships. In the experiment that is properly designed and carried out, every step is carefully planned to limit effects to the variables of interest. Few variables are studied, so that the problem of disentangling complex relationships does not exist. Sys- tematic relationships between changes in some variables and consequences for other variables are established so that the experimenter can say “If X, then Y.” Full details of the experimental procedure are reported so that the results can be replicated by investigators in other laboratories.

Psychologists who are critical of laboratory research suggest that too often such research is artifi cial and has limited relevance to other contexts. The sug- gestion is that what works in the laboratory may not work elsewhere. Further- more, although relationships between isolated variables may be established, such relationships may not hold when the complexity of actual human beha- vior is considered. Also, since laboratory research tends to involve relatively brief exposures to stimuli, such research may miss important processes that occur over time. As you read about personality research in the subsequent chapters of this book, a question to ask yourself is how successful the different theories are in establishing experimental fi ndings that generalize to real-world situations.

As a human enterprise, experimental research with humans lends itself to infl uences that are part of everyday interpersonal behavior. The investigation of such infl uences might be called the social psychology of research. Let us consider two important illustrations. First, some factors infl uencing the be- havior of human subjects may not be part of the experimental design. Among such factors may be cues implicit in the experimental setting that suggest to the subject that the experimenter has a certain hypothesis and, “in the interest of science,” the subject behaves in a way that will confi rm it. Such effects are known as demand characteristics and suggest that the psychological experi- ment is a form of social interaction in which subjects give purpose and mean- ing to things (Orne, 1962; Weber & Cook, 1972). The purpose and meaning given to the research may vary from subject to subject in ways that are not part of the experimental design and thereby serve to reduce both reliability and validity.

Complementing these sources of error or bias in the subject are unintended sources of infl uence or error in the experimenter. Without realizing it, experi- menters may either make errors in recording and analyzing data or emit cues to the subjects and thus infl uence their behavior in a particular way. Such un-