The major instrument used in the study is the Statistical Literacy Interest Measure (SLIM). It contains 16 items from a larger interest inventory of 30 items, shown as items R1 through to R36 in Appendix A. The development of the items in the interest inventory and the subsequent development of SLIM are described in Chapter 5, whereas final results are reported in Chapter 6.
Self-efficacy for statistical literacy (SESL) scale
Given the expected close association between interest and self-efficacy, the second major instrument used in this study is the Self-efficacy for Statistical Literacy (SESL) scale, shown in Appendix A as items S41b through to S50c. As with SLIM, the initial development of this instrument is described in Chapter 5 and final results are reported in Chapter 6.
Demographic and other data
All students were asked to provide some demographic data. These included their age, their gender and their year level at school. Students in the second
and final stages were also asked to provide their names so that achievement data could be linked to their interest data.
Students in the final stage were asked four questions regarding the frame of reference they used when making interest assessments. These questions, shown as items IE42 to IE45 in Appendix A, were answered as self-descriptions with the existing five-point Likert scale. The first self-description, worded “compared to others in my class I am good at maths,” sought to assess the extent to which students used an external frame of reference. The second self-description, worded “out of all my subjects I usually get my best marks in maths,” sought to assess the extent to which students used an internal frame of reference. The third self-description, worded “I find statistics more interesting than other work we do in maths,” sought to assess the extent to which students compared their interest in statistics with their interest in other areas of
mathematics. The last self-description, worded “the statistics that I do in maths classes is more interesting than the statistics that I do in other subjects,” sought to assess the extent to which students compared their interest in the statistics encountered in mathematics classes, with the statistics encountered in other classes.
Achievement data
Teachers of students in the second and final stages of the study were asked to provide a rating of their students’ mathematics achievement. Teacher ratings of student achievement are known to be strongly predictive of actual student achievement (Egan & Archer, 1985) and display high levels of validity (Hoge & Coladarci, 1989). The teachers in this study were asked to rate each student on a five point scale from A, the best category of achievement, to E, the worst category of achievement. This A to E assessment category is used throughout Australia, having been mandated by the Australian Government (Department
of Education, Science and Technology, 2005). Of the 570 students participating in these two stages, achievement data were available for 452. The distribution of their grades is shown in Table 4.3.
Table 4.3
Distribution of mathematics grades (Maths-grade)
Category Frequency Percent A grade 116 25.7 B grade 190 42.0 C grade 107 23.7 D grade 29 6.4 E grade 10 2.2 Total 452 100.0
In order to control the influence of classroom factors on achievement, a relative mathematics grade was also considered. More specifically, the student’s grade relative to the median grade of his or her class was determined. As an example, a student with a maths grade of B in a high-ability class where the median grade was A, was assigned a below median grade. These adjustments resulted in a three category structure that is shown in Table 4.4. Although this variable enabled classroom factors to be controlled, the resulting three category structure resulted in an unavoidable loss of statistical power (Manor, Matthews, & Power, 2000).
Table 4.4
Distribution of relative mathematics grades (RelMaths-grade)
Category Frequency Percent Below median grade 120 26.5
Median grade 227 50.2
Above median grade 105 23.2
A measure of students’ statistical literacy knowledge (SLK) was also available from some of those students in this study who attended StatSmart schools. Students in these schools who were actually involved in the StatSmart project completed a series of tests that assessed their knowledge of statistical literacy. More specifically, upon entering the project students completed a pre-test, approximately six months later they undertook a post-test, and finally 12 months later they completed a longitudinal test. The items used in these tests and the method used for scoring these items, are detailed in Callingham and Watson (2005). Further details regarding the methodology used in the StatSmart project are described in Callingham and Watson (2007).
Of the 483 students in this study attending StatSmart schools, 188 did not complete a StatSmart test. Such students were recruited by the teacher from classes that they had not nominated for participation in the StatSmart project. Teacher motives for including or not including classes of students in the study are unknown. It is unlikely, however, that the high proportion of missing data in this instance would adversely influence the study’s results. As a result of these missing data, SLK scores were only available for 295 students. Of these students, 161 completed their StatSmart tests at the end of the first year of the study with the remainder completing theirs at the beginning of the next year. Seventy-one of the students who completed their StatSmart tests at the end of the first year of the study completed their interest assessment approximately six months later in the first half of the second year of the study. During this
intervening period, however, summer holidays occurred making it unlikely that their interest in statistical literacy would have changed significantly.
Teacher influences
As discussed, students who completed a StatSmart test did one of three tests: a pre-test, a post-test, or a longitudinal test. The type of test students did,
therefore, is a variable that represents a measure of how long students were in a StatSmart school. In many cases it also represents a measure of how long they were in a class taught by a StatSmart teacher, in that students in the class of a StatSmart teacher did a pre-test near the beginning of the school year and a post-test near the end of the year. Students who did the longitudinal test, which was administered one year later, may not have been with the same teacher, but were in the same school. Given that it was the teachers who were directly involved in the intervention, this variable represents a measure of the influence of the teacher and/or school over and above other individual factors. Of the 295 students in this study who completed StatSmart tests, 49% did the pre-test, 32% did the post-test and the remainder did the longitudinal test.
Variables used during modelling
In order to answer the research questions, a number of variables were created that reflect the data described above. A summary of these is shown in Table 4.5.
Table 4.5
Summary of instruments and associated variables
Instrument Assessment method Variable SLIM Rasch-scaled student responses Interest SESL Rasch-scaled student responses Self-efficacy StatSmart
tests
Rasch-scaled student responses SLK Achievement
data
Teacher obtained estimate from A to E
Maths-grade Achievement
data
Teacher estimate relative to class median grade
RelMaths- grade StatSmart
tests