The rise and development of construct validation

All the theorists of educational measurement who write about the development of validity theory (e.g. Angoff 1988, Anastasi 1986, Messick 1989a, Moss 1992, Shepard 1993) ascribe the introduction of the modern concept of construct validation to Cronbach and Meehl (1955). Much more clearly than earlier writers, these authors focused on the meaning of the scores rather than prediction as the essential question in validation. Angoff (1988:26) recounts that ”in construct validity … Cronbach and Meehl maintained that we examine the psychological trait, or construct, presumed to be measured by the test and we cause a continuing, research interplay to take place between the scores earned on the test and the theory underlying

the construct.” In other words, the validation of a test was intimately bound with the theory of the trait or ability being measured. When construct validity was defined in this way, it could not be expressed by a single coefficient. It had to explain the meaning of the test scores. This required combining empirical evidence with theoretical statements about what the scores stood for.

Cronbach and Meehl’s original (1955) concept of construct validity was expressed in the language of positivist philosophy of science dominant at the time. A construct could only be accepted if it was located in a fully specified ”nomological network”, which defined its relationships with other constructs and with practical observations by clear causal or statistical laws. The positivistic emphasis on the logical structure of scientific theories has since given way to instrumentalism and realism in validation research as in social sciences generally (on this development, see e.g. Messick 1989a:22- 30, Cronbach 1989:158-163). The focus is more on the way in which scientific inquiry is conducted, and very similar guidelines are provided although they differ in terms of how they view truth, whether it is metaphysical and viewer-dependent or whether it exists in the world independently of any viewers.

Apart from philosophical changes, Cronbach (1975, 1986, 1988) has also argued that his and Meehl’s call for fully specified relationships was both unrealistic and impractical. Less clearly specified constructs can be highly useful to explain what test scores mean. Several modern theorists of educational measurement concur with this view (e.g. Anastasi 1986, Messick 1989a, Wiley 1991). Shepard (1993:417) continues: ”Nevertheless, by some other name the organizing and interpretive power of something like a nomological net is still central to the conduct of validity investigations. Perhaps it should be called a conceptual network or a validity framework.” This view of the continuing importance of the conceptual network for validation is important for the present thesis. It shows how central the construct definition is in the current concept of validity.

The new formulation of construct validity as a theoretical concept was elaborated further when Campbell and Fiske (1959) presented a conceptual and empirical test which guided its operationalization. To pass the test, validators would have to assemble ”convergent evidence, which demonstrates that a measure is related to other measures of the same construct and other variables that it should relate to on theoretical grounds, and discriminant evidence, which demonstrates that the measure is not unduly related to measures of other distinct constructs” (Moss 1992:233). Thus, as Angoff (1988:26) explains, an empirical design to investigate a

proposed construct would involve tests of two or more constructs tested through two or more different methods. The results might support the theory and the tests, or they might call into question the tests or test methods, the hypothesized relationship between the constructs, or the theories governing the constructs.

The logic with the call for convergent and discriminant evidence is that it offers proof both for what the test scores reflect and for what they do

not reflect. In a further development in the theory of construct validity, Cronbach (1971) applied the same logic on a higher level of abstraction when he proposed that construct validity be guided by a search for plausible rival hypotheses. To defeat these would offer the strongest possible support for the current theory. This parallelled Popper’s (1968) view concerning the development of scientific theories, as Cronbach acknowledged. Concurring with and promoting this theoretical development, Messick (1980) called the search for plausible rival hypotheses the hallmark of construct validation.

Messick (1989a:18-19) gives a detailed account of how the testing field has moved towards recognising the unitary nature of validity. He describes how the Standards move from an explicit division of validity into types in 1966 and 1974 to the 1985 version where validity is a unitary concept and ”an ideal validation includes several types of evidence” (APA 1985:9). He points out that similar developments can be seen in successive editions of major textbooks on testing theory such as Anastasi’s

Psychological Testing and Cronbach’s Essentials of Psychological Testing.

Messick himself was an early defender of the unified view. In 1980 (p. 1015) he argued that ”construct validity is indeed the unifying concept of validity that integrates criterion and content considerations into a common framework for testing rational hypotheses about theoretically relevant relationships.” Agreeing, Cronbach (1990:152) concludes: ”The three famous terms do no more than spotlight aspects of the reasoning. To emphasize this point the latest Standards speak not of ’content validity,’ for example, but of ’content-related evidence of validity.’ The end goal of validation being explanation and understanding, construct validation is of greatest long-run importance.” Defined in this way, the goal of construct validation is that of science in general.

The development of validity theory in language testing has parallelled the development in educational measurement. Early textbooks such as Lado (1961) and Harris (1969) discuss internal and external validity as per Loevinger’s (1957) model. In the 1970s, authors such as Davies (1977) and

Heaton (1975) considered validity in terms of four distinct types: content, predictive, concurrent, and construct. They also discussed and dismissed face validity like measurement theorists in general. Oller (1979) did not discuss validity as a theoretical concept explicitly, but his references to validity (eg. 1979:417-418) show that he considers it mainly a correlational indicator and dependent on reliability. The theoretical debate around his Unitary Competence Hypothesis (reacted to eg. by Bachman and Palmer 1982 and Upshur and Homburg 1983) was fought on empirical, quantitatively analysed evidence for construct validity. More recently, Bachman (1990) brought validity theory up for a thorough discussion and promoted the unified theory of construct validity proposed by Messick. Cumming (1996:5) summarises the development: “Rather than enumerating various types of validity ... the concept of construct validity has been widely agreed upon as the single, fundamental principle that subsumes various other aspects of validation ... relegating their status to research strategies or categories of empirical evidence by which construct validity might be assessed or asserted.” When Bachman and Palmer (1996:21) list the essential properties of language tests, they mention and define construct

validity: “Construct validity pertains to the meaningfulness and appropriateness of the interpretations that we make on the basis of test scores.”

To summarise, the shift from prediction of specific criteria to explanation of the meaning of test scores as the aim of validation raised construct validity to a central position in validation inquiry. At the intermediate stage, theorists identified three or four types of validity of which construct validity was one, but several theorists made compelling cases that the other ”types” of validity could not sustain a validity argument alone and were better seen as aspects of a unified theory of construct validity.