RELIABILITY,
VALIDITY, ETHICAL
BEHAVIOR
c02TheScientificStudyofPeople.indd Page 43 07/11/12 7:18 PM user-019A
variable, or construct, that it purports to measure (Cronbach & Meehl, 1955; Ozer, 1999). To establish that a test possesses construct validity, personality psychologists generally try to show that the test relates systematically to some external criterion, that is, to some measure that is independent of (i.e., external to) the test itself. Theoretical considerations guide the choice of an external criterion. For example, if one were to develop a test of the tendency to experi- ence anxiety and wanted to establish its construct validity, one would use theoretical ideas about anxiety to choose external criteria (e.g., physiological indices of anxious arousal) that the test should predict. One generally would establish validity by showing that the test correlates with the external criteri- on. However, in addition to correlational data, tests of validity might involve comparisons of two groups of people who are theoretically relevant to the test. A group of people who have been diagnosed by clinical psychologists as suffer- ing from an anxiety disorder, for example, should get higher scores on the purported anxiety test than people who have not been so diagnosed; otherwise one obviously would not have a valid test of anxiety.
There are other aspects of validity (Ozer, 1999; West & Finch, 1997). For example, if one is proposing a new personality test, one should be able to dem- onstrate that the test has discriminant validity: It should be distinct, empiri- cally, from other tests that already exist. If, hypothetically, one proposes a new test of “worrying tendencies” and fi nds that it correlates extremely highly with existing tests of neuroticism, then the new test is of little value because it lacks discriminant validity.
A relatively new idea about test validity ties the concept of validity to the concept of causality. A test, in this view, is a valid measure of a psychologi- cal quality if (a) that quality actually exists and (b) variations in the quality causally infl uence the outcomes of the measurement process (Borsboom, Mellenbergh, & van Heerden, 2003). Here’s an example. Suppose the quality you want to measure is “skill in solving everyday social problems” (e.g., problems such as fi guring out how to have more friends or save more mon- ey). A valid measure might be the number of solutions people can generate when presented with everyday problems to solve (Artistico, Orom, Cervone, Krauss, & Houston, 2010). It is valid because it fi ts both criteria: (1) The attribute exists: All individual people possess some level of skill in solving everyday problems; (2) Variations in the attribute cause variations in the outcome: A lower level of skill (less knowledge of problem-solving strategies and less ability to put that knowledge into practice) causes people to gener- ate fewer solutions. Contrast this example with a hypothetical one: a mea- sure of the infl uence of ghosts on personality functioning. (The measure might contain questions such as “How many times in the past month has your personality been affected by ghosts? 1–3 times? 4–10 times? > 10 times?). No matter what people say in response to the test, and no matter what the correlation between test responses and other outcomes, the test is not a valid measure of the construct, in this new view. Why not? Because the at- tribute (ghosts and their infl uence) does not exist. Since it doesn’t exist, it cannot exert a causal infl uence on test responses. Thus there can be no valid measure of this construct, in a causal account of test validity.
In sum, reliability concerns the questions of whether a test provides a sta- ble, replicable measure, and validity concerns the questions of whether a mea- sure actually taps, and is infl uenced by, the psychological quality it is supposed to be measuring.
GOALS OF RESEARCH: RELIABILITY, VALIDITY, ETHICAL BEHAVIOR 45 THE ETHICS OF RESEARCH AND PUBLIC POLICY
Research in psychology is laden with ethical concerns. Ethical issues pervade both the conduct of research and the analysis and reporting of research results (Smith, 2003). These concerns are long standing. A half-century ago, in a famed line research, participants in the role of “teachers” were instructed to teach other participants (“learners”) a list of paired associate words and to punish them with an electric shock when they made errors on the word list (Milgram, 1965). Although actual shock was not used, the “teachers” believed it was. Many administered high shock levels despite the learner’s pleas for them to stop. In another study, participants lived in a simulated prison envi- ronment in the roles of guards or prisoners (Zimbardo, 1973). “Guards” ver- bally and physically abused the “prisoners,” who allowed themselves to be treated in a dehumanized way. In both studies, participants experienced such severe levels of stress that one must question whether the gains to science outweighed the costs to the participants.
Such research programs raise fundamental questions about the ethics of research. Do experimenters have the right to deceive research participants? To place them under signifi cant stress? These, in turn, raise a broader question: What ethical principles guide answers to such questions?
The American Psychological Association (APA) has adopted a set of ethical principles (American Psychological Association, 1981). Their essence is that “the psychologist carries out the investigation with respect and concern for the dignity and welfare of the people who participate.” This includes evaluating the ethical acceptability of the research, determining whether subjects in the study will be at risk in any way, and establishing a clear and fair agreement with research participants concerning the obligations and responsibilities of each. Although deception is recognized as necessary in some cases, it must be minimized. Researchers always bear a responsibility to minimize participants’ physical and mental discomfort and harm. In addition to the APA guidelines, similar federal guidelines (that is, within the United States, guidelines formu- lated by a branch of the U.S. federal government) guide research. All research projects in psychology must be reviewed and approved by an ethics board that evaluates whether the research adheres to these guidelines.
As noted, ethical principles also apply to the reporting of research results. A long-standing concern is “the spreading stain of fraud” ( APA Monitor, 1982)—that is, the possibility that the researcher’s reporting of results is not accurate but, instead, has been distorted by his or her personal motives. In the 1970s, statistical analyses indicated that Sir Cyril Burt, a once prominent British psychologist, intentionally misrepresented data when reporting re- search on the inheritance of intelligence (Kamin, 1974). Early in the 20th century, a researcher was forced to retract from the scientifi c literature a previously published study because it did not accurately report valid research results (Ruggiero & Marx, 2001). More recently, a psychologist resigned from his job after admitting that data in multiple studies of his were entirely fabricated ( New York Times, November 2, 2011).
Fraudulent research reports are rare. Yet, in psychology or any science, fraud is not impossible. Science’s safeguard against fraud is independent rep- lication, that is, replication of results by a researcher other than the one who ran the original study. A large percentage of the results you’ll read about in this book have been replicated independently.
c02TheScientificStudyofPeople.indd Page 45 07/11/12 7:18 PM user-019A
Much more subtle than fraud are personal and social biases that affect how scientifi c questions are developed and what kinds of data are accepted as evi- dence (Pervin, 2003). In the study of sex differences, for example, researchers might pose questions in a manner that is gender biased (e.g., asking whether “women are as skillful as men” on a task) or might be more likely to accept the validity of research results that fi t their preexisting expectations about men and women. Although scientists strive to remain objective, they—just like any- one else—may sometimes fail to recognize how their personal opinions and expectations affect their judgments and conclusions.
The ethical reporting of research in personality psychology is important not only to advances in science, but to society at large. Personality research is ap- plied in numerous domains: clinical treatments for psychotherapy; educational policies to motivate students; tests to select among applicants for jobs; and so forth. These applications heighten the research psychologist’s responsibility to report research accurately and comprehensively.
All personality scientists hope to obtain research results that are reliable and valid, as you learned above. They differ, however, in the strategies through which they try to achieve that goal. Three overarching research strategies predominate in the fi eld: (1) Case Studies; (2) Correlational Studies; and (3) Experiments. Let’s introduce these three strategies now. You’ll see them again and again in later chapters.
CASE STUDIES
One strategy is to study individual persons in great detail. Many psychologists feel that in-depth analyses of individual cases, or case studies , are the best way to capture the complexities of human personality.
In a case study, a psychologist interacts extensively with the individual who is the target of the study. In these interactions, the psychologist tries to develop an understanding of the psychological structures and processes that are most important to that individual’s personality. Using a term introduced previously, case studies inherently are idiographic methods in that the goal is to obtain a psychological portrait of the particular individual under study.