It is the application of the partial credit Rasch model to cognitive developmental data that has paved the way for original conceptions of how fundamental measurement principles might be meaningfully applied in unexpected settings. Let us take a well-known problem from Piaget's work, often called conservation of matter, an interview task often used with, say, 3- to 7-year-olds, and therefore
< previous page
page_89
next page >
quite unsuitable for a written task format. A child judges two balls of playdough to have the same amount. Then one of them is rolled out into a snake or sausage shape right in front of the child's eyes. The child then is asked to judge again. The question is something like, "Does the snake have more playdough than the ball; does the ball have more playdough than the snake; or do they have the same amount of playdough?" Anyone who has watched young kids share a cupcake, pour drinks for friends, or gladly swap little coins for bigger ones will know that whereas adults often do not "see the problem" here, it is only during the grade school years that children grow to understand that the amount of playdough, amount of juice, or number of sweets will remain invariant no matter how the objects are rearranged.
Piaget's approach was to claim that the child who consistently conserves shows qualitatively superior thinking to the child who does not, backing his claim with detailed logical analyses of the child's supposed thinking patterns. When we psychologists first tried to quantify our replications of this research, we adopted 0 = couldn't conserve, 1 = could
conserve as the task performance summary, which hindsight has shown to be rather na_. The extent of the trivialization that routinely went on can be imagined: Little Johnny gets inter-viewed for 15 min on three conservation tasks (amount of playdough, amount of juice, and number of sweets each suitably rearranged) and gets scored like this: 1, 1, 0
(conserves playdough and juice but misses number). Betty and the psychologist chat and play together for 20 min, and she gets scored 0, 1, 0; whereas Jane gets 1, 1, 1 after a mere 10 min of concentrated effort on her part. After an equally brief encounter, it seems Bill is enchanted with the magic that makes things more or less, just by moving them around (0, 0, 0). Then we would compound our errors by adding the scores up for each child: 2 for Johnny, 1 for Betty, and 3 for Jane, whereas Bill gets 0 however you look at it. Often we would take this even one step further and, claiming that 2/3 was enough to describe Johnny and Jane as "conservers" and Betty and Bill as "nonconservers." Consequently, if the purpose of the research was to relate cognitive development in kids to their mathematics scores, there was no meaningful result, which is not surprising!
Seen through the lens provided by the partial credit model, the opportunities for serious and sensitive measurement seem to multiply very productively. First, we could see that it would not be necessary to have just 0 and 1 scoring. We could also have 0, 1, 2 or 0, 1, 2, 3 if we found suitable criteria to score against. Not only could we have "items" with two or three steps (i.e., not the dichotomous model), but we also were not constrained to the same number of steps for each item (i.e., not the rating scale model). Even more, we could mix dichotomous and polytomous items in the one test. Moreover, instead of providing a single overall score for each task or complete interview, we could see each task addressing a number of key aspects, each of which could be scored. Thus, the playdough task could be broken down into the following subtasks: (a) judges
< previous page
page_90
next page >
Page 91
initial equivalence: no = 0, yes = 1; (b) conserves after snake transformation: no = 0, yes = 1; (c) uses "longer"
appropriately: never = 0, sometimes = 1, consistently = 2; (d) gives reasons based on perception = 0, by rolling the snake into a ball = 1, saying "You didn't add or take away anything" = 2, claiming "It's always the same no matter what you do" = 3.
Therefore, for the string of criteria or items a, b, c, d, for just the first test (playdough task), we could end up with Johnny's scores of 1, 1, 2, 2; Betty's 1, 1, 1, 1; Jane's perfect response string 1, 1, 2, 3; and Bill's 1, 0, 0, 0. It can be seen how we have now discriminated differences between Betty and Bill (both received 0 for playdough with dichotomous scoring), and between Jane and Johnny (both scored 1 in the dichotomous situation). Remember the ordered data matrix from chapter 2? We can do it again (see Table 7.1).
In Table 7.1, we see the same sort of evidence for developmental sequences that we saw in the sorted dichotomous data in Table 2.2. Being on guard about making unwarranted inferences from the data, we merely observe that we have recorded ordered increases in response levels (i.e., 0 = 1 = 2 = 3, etc.). Of course, we also refrain from drawing unwarranted equivalences between the values of 1 for items a and b, between the scores of 2 for items c and d, and so forth. All we are entitled to say from these data categorizations is that for item a, 0 = 1, and for item b, 0 = 1. We may not claim that 1 (on a) = 1 (on b), or that 2 (on c) = 2 (on d). We would use Rasch modeling to estimate those relations, the intervals between those ordered values, during the analysis process.
The data used to demonstrate the use of the partial credit Rasch model in this chapter have been chosen for a number of important reasons. We could have chosen a routinely used written mathematics or science test, an essay scoring guide, or a medical rehabilitation example, as long as it offered the possibility of getting answers partly correct. Many scoring situations would do, as long as the grading principle represented by the following responses is implemented: "wrong- partly correct-right" or "fail-some progress towards mastery-more complete response-mastery." However, with the current example, we can learn something about the partial credit model while we open our eyes to the range of possibilities for quantifying what traditionally has been seen as qualitative data in the human sciences.
TABLE 7.1
Ordered Data Set for four Children on Four Polytomous Items
Criteria a b c d
Bill 1 0 0 0
Betty 1 1 1 1
Johnny 1 1 2 2
Jane 1 1 2 3
< previous page
page_91
next page >
Shayer, Küchemann, and Wylam (1976) from the University of London had a number of reasons for developing the Piagetian Reasoning Task (PRTIII-Pendu-lum) for classroom use. The PRTIII-Pendulum we used in chapter 5 was one of the demonstrated class tasks that they developed to replace administration using the traditional Piagetian interview technique. They wanted data collection de-vices that could be administered to whole classes at a time, whereas Piaget's technique was a one-on-one interview. They wanted tests that could be used by interested school science teachers, whereas Piaget claimed that his interviewers needed 1 year of daily practice to become competent. They wanted tests that could yield quantifiable data, whereas Piaget's work was notorious for its un-compromisingly qualitative approach. Well, the Rasch model cannot change the Piagetian requirements for superior interview skills or make it possible to interview 30 kids simultaneously, but it can make wonderfully quantitative what is regarded traditionally as qualitative data. It can construct interval scale measures on which a whole range of the usual statistical techniques can be
meaningfully employed. Indeed, the following piece of research, first reported in Bond and Bunting (1995), was inspired exactly by the existence of the partial credit Rasch model (PCM).
CLINICAL INTERVIEW ANALYSIS: