ASSESSING COGNITIVE AND LEARNING STYLES

The assessment of cognitive and learning styles is undoubtedly the Achilles heel of the concept. In a review of the area, Irvine (2001) stated rather dis- appointedly that “the enforced conclusion one may have to accept with reluctance is that the means of pursuing, in operational form, the elusive pimpernel of an acceptable measurement protocol for style is not available” (p. 274). He found this all the more disconcerting as in their everyday lives people do not seem to have any trouble identifying various style characteris- tics. As he pointed out, “the notion of style is so intuitively certain in ordi- nary people untrammeled by psychologists’ preoccupations with meas- urement, that professional entertainers make a good living by mimicking styles among the great, the good, the bad, and the ugly” (p. 274). So, if this claim is true and style is relatively easy to capture and imitate, why is it so difficult to measure?

We can answer this question in at least two ways. On the one hand, we must recognize that there are some style instruments that appear to do a rea- sonably good job. Two of the best known ones, Riding’s Cognitive Styles Analysis (CSA) and Kolb’s Learning Style Inventory (LSI), are examined in some detail next. On the other hand, we may also want to consider Chapelle’s (1992) summary: “I believe that the most important and relevant human constructs are those which are neither interesting to ‘authorities’ nor measurable at present” (p. 381). That is, it could be the case that although learning styles are valid and important psychological entities, measurement theory has not as yet developed the right methodology to capture them. In the history of physics, for example, we find several examples when a theory was proposed well before the adequate measuring procedures and instru- ments had been developed to verify it.

When it comes to cognitive and learning styles, currently we know only of two established ways of assessing them: either by relying on the learners’ own self-reports on how they perceive their cognitive functioning, or by asking learners to perform mini-information-processing tasks and then making inferences from their performances. Kolb’s LSI is a good example for the first type and Riding’s CSA for the second (the Embedded Figures Test, which has traditionally been used to measure field dependence–inde- pendence, is also a test of performance belonging to the second category and it will be discussed later in this chapter). Interestingly, Rebecca Oxford and her colleagues have been experimenting since the 1990s with a qualitative approach to infer language learners’ style preferences by eliciting thematic student essays in several locations around the world and then submitting these narratives to content analysis (e.g., Oxford, 1999d; Oxford, 2001; Oxford & Massey, in press). This is a real innovation in the area of style assessment, resulting in fresh insights into the learners’ perceptions of the impact of their styles, as well as the mismatches between their and their teachers’ styles, on their learning process (see later in more detail).

Kolb’s Learning Style Inventory (LSI)

The original LSI instrument was a nine-item self-description questionnaire. Each item asked the respondent to rank-order four words in a way that best described their learning style. One word in each item corresponded to one of the four learning modes—concrete experience (e.g., “feeling”), reflective observation (e.g., “watching”), abstract conceptualization (e.g., “thinking”), and active experimentation (e.g., “doing”). The most recent version of the instrument, Version 3 (Kolb, 1999), has 12 items and the actual wording has been changed from single words to short statement format, as illustrated in Table 5.2.

The initial validation of the LSI scales was carried out with a sample of 1,933 participants. As Kolb (1984) reports, the theoretical assumption that the ‘abstract’ and ‘concrete thinking’ categories were opposite ends of a continuum was born out by significant negative correlation (-0.57) between the two orientations. Similarly, there was also a significant negative correla- tion (-0.50) between ‘active’ and ‘reflective information processing’ orien- tations. On the other hand, there was no substantial intercorrelation between the components associated with the two different dimensions.

Although the LSI scales are theoretically well-founded and have good psychometric properties, the big question still remains: Are the attributes that the scales measure indices of learning styles or something else? Kolb et al. (2001) offered some evidence of the ambiguous nature of this issue be- cause, as they summarized, the main dimensions of the LSI correlate signifi-

cantly with certain components of the Myers-Briggs Type Indicator (MBTI), which is primarily a personality type inventory, although as was pointed out in chapter 2, various psychological types display a strong link with certain learning styles and therefore the MBTI is often cited when discussing learn- ing styles (cf. Ehrman, 1996).

A further problem with self-report measures such as the LSI is that the actual items usually focus on behavioral correlates of assumed style charac- teristics (Riding, 2000b); in other words, respondents are not asked about their style features but rather about typical behaviors associated with these style features. The item “I write lists of everything I need to do each day” from the Learning Style Survey (Cohen, Oxford, & Chi, 2002; see Table 5.5) is a good example of this. While behavioral self-report items are not necessarily an inappropriate way of obtaining an index of an underlying trait, problems start when researchers correlate the test results with behav- ioral criterion measures, such as the learners’ performance. As Riding pointed out, this creates a circularity of correlating, in effect, behavior with behavior, in contrast to identifying fundamental sources of style that can be seen to affect behaviors.

Table 5.2. Sample items from Kolb’s (1999) Learning Style Inventory (Version 3: LSI3)

The four statements in both sample items need to be rank-ordered accord- ing to how they refer to the respondents. Thus, four marks are to be given to the statement that is most true and one to the one that is least appropriate. When I learn:

____ I like to deal with my feelings ____ I like to watch and listen ____ I like to think about ideas ____ I like to be doing things I learn best from:

_____Observation

_____Personal relationships _____Rational theories

Riding’s Cognitive Styles Analysis (CSA)

Currently the CSA (Riding, 1991) appears to be one of the most accurate instruments to measure styles for a number of reasons: First, it focuses on cog- nitive styles rather than learning styles, which allows it to target a narrower and more precisely definable domain. Second, it does not utilize the introspective self-report format that the LSI is an example of, but rather tests respondent performance directly. Third, the reliability of the test is greatly enhanced by the fact that it is computer-based. The CSA assesses both ends of the wholist-ana-

lytic and verbal-imagery dimensions, and comprises three subtests:

• Subtest 1, Verbal-Imagery dimension: Students are presented a number of statements (48 in total), one at a time, which require a simple true or false response by pressing a button on the keyboard. Half of the state- ments are about conceptual categories (e.g., “table and chair are the same type”); the other half describe the appearance of objects (“snow and chalk are the same color”). Half of the statements of each type are true, the other half false. This subtest is based on the assumption that

imagers respond more quickly to visual items because they find it easier

to represent the information in terms of visual images, whereas verbaliz-

ers are at an advantage with the conceptual items because the conceptual

category membership is verbally abstract in nature and cannot be represented in visual form. The computer automatically records the response time to each statement and uses this information to calculate a ratio of verbal response time to visual response time. A low ratio corre- sponds to a verbalizer and a high ratio to an imager, with the intermedi- ate position being described as bimodal. Because both types of item require reading, factors such as reading speed and ability are inherently controlled for by the calculation of the ratio.

• Subtest 2, Wholist dimension: Students are presented pairs of complex geometrical figures side by side on the screen (a total of 20 pairs) and they have to decide about each pair whether they are identical or not.

Wholists are assumed to respond more quickly because their natural ten-

dency to focus on the whole picture corresponds to the task of absorbing the whole shapes.

• Subtest 3, Analytic dimension: This subtest is similar to the previous one in presenting a pair of geometrical shapes at a time (20 times), but this time the question is whether the first figure, which is a relatively simple geometrical shape (e.g., a square or a triangle), is contained within the second, more complex figure. Analytics, who are more inclined to focus on details, respond more quickly because the task requires the larger shape to be broken down into its constituent parts. Once again, the com-

puter records the response times and calculates the wholist–analytic ratio.

Throughout the test the testees are not made aware that the assessment uses response time, because the intention is that they undertake the tasks in a relaxed way that reflects their usual manner of processing information. And because the final indices are based on ratios, the actual response speed does not influence the style result.

Riding and Rayner (1998) emphasized several positive features of the CSA: (a) It is an objective test in the sense that it is objectively scored and the respondents are not aware of the real focus of the assessment; (b) both ends of the style continuums are assessed, which makes it distinct from measuring abilities; (c) because of the limited and simple language it in- volves, its use is versatile across age and proficiency groups; and (d) the computerized format creates a context-free character, which allows it to be used across situations and cultures. Furthermore, Riding (2001) reported statistical evidence that the two dimensions are unrelated to one another and show no age or gender differences. What is just as important, the scales ap- pear to be unrelated to intelligence, which supports the fact that the styles measured are not simply subtypes of ability. Finally, although correlations of some magnitude were found between certain personality dimensions and the CSA scales, the overall pattern appeared to point to a model in which physiologically based personality sources are independent of cognitive style but are moderated by style in their effect on behavior.

Recently, Peterson, Deary, and Austin (2003a) examined the reliability of the CSA by comparing performance on the original CSA test and a new parallel version. They concluded that in its present form the test was not suf- ficiently reliable or internally consistent. However, the authors added that when the CSA was doubled in length, the wholist-analytic dimension of cognitive style preference became a more stable and reliable measure. Not surprisingly, Riding (2003) questioned these findings because he claimed that Peterson et al.’s study was not executed properly. And again unsurpris- ingly, Peterson et al. (2003b) maintained that their original concerns were valid.