The instruments

4.2.3.1 Initial and final assessment

The first part of the initial assessment consisted of completing the ‘list of concepts.’ The list had been tested earlier in an extensive try-out in four groups with a total of 63 students (appendix 2B). The students were asked to indicate for each concept whether they knew what the concept meant, whether they could tell a teaching narrative that contained that concept and from which categories (own teaching practice, video/film, literature, lectures/workshops) the narrative was taken. During the course the list provided guidance and insight into progress. At the end the students were asked which concepts had gained meaning for them and for which they could provide a fitting ‘narrative.’ For the teacher educator the yield of the list of concepts at the start of the course was an indication which concepts would need more attention. The yield for the researcher was extra information about the theoretical knowledge the students assumed (un)familiar for themselves at the start and at the end of the course.

The second part of the initial assessment was meant to see at the start of the course meetings how students described practical situations and to what degree they used theory. For this purpose, after a first introduction of The Guide, the students were given the assignment to write a reflective note of one typed page for one of the twenty-five narratives in The Guide, with the narrative to be chosen freely. Further data were taken from video observation of the group of students during the discussion based on these reflective notes. This part of the initial assessment yielded two types of data. In the first place the number of theoretical concepts and theoretical notions each student used in doing the assignments, and secondly statements from students using theoretical concepts or notions of theoretical concepts.

The final assessment consisted of three parts, namely filling in the ‘list of concepts,’ writing two teaching narratives for two (newly) familiar concepts (e.g., appendix 5), and writing a reflective note on a teaching situation in MILE that had not been used in that

Theory-enriched practical knowledge in mathematics teacher education

course (e.g., appendix 6). This final reflective note had an essential function in the study for comparing the use of theory by the students.

The list of concepts in the final assessment differed from that in the initial one in the sense that students were now asked to indicate which concepts had become familiar (appendix 2A,B).

The two teaching narratives gave information to the researcher on how the students gave meaning to the two theoretical concepts. The narratives also indicated whether the student made a connection with other theoretical concepts.

The reflective note provided information on the nature of using theory by the student teachers (factual description, interpretation, explanation or responding) and the level of use of theory (number of concepts; number of units with meaningful relationships between concepts) (see section 4.3.9).

4. 2.3.2 Observation

All meetings were recorded on video tape. The video material was used to analyze the discussion for the use of theory by students, to make an inventory of interventions by the teacher educators and to obtain other important data on the use of theory and for setting up the learning environment for the large scale study. Transcripts of the video recordings were made.

4.2.3.3 Video stimulated recall

This study used a variant of the stimulated recall procedure. During a stimulated recall interview (Krause, 1986; Verloop, 1989) the students made explicit their thinking in reaction to watching video sequences of the discussion in which they participated. In the penultimate lesson, during the so-called ‘game of concepts’ (section 4.2.2.1), students, directed by the teacher educator, discussed whether there was a demonstrable connection between given theoretical concepts and four practical situations. The video recordings of these discussions were used for stimulated recall sessions with individual students after that meeting. The students were given some general instructions before the interview30_.

The approach of the stimulated recall procedure in this study differs slightly from the standard procedure; this concerns the time interval (max. 4 days) between video recording and interview, as it was not always possible to have the interview immediately following the recording.

4.2.3.4 Concept mapping

This study used the technique of concept mapping (Novak, 1990; Morine-Dershimer, 1993; Zanting, Verloop & Vermunt, 2003) to verify to what degree students were able by the end of the course to make connections between ten theoretical concepts that had come up in the course31_{. To this purpose the teacher educator asked them to each}

own insight. The assignment was: “Order the cards according to your own insights and glue them on the large sheet of paper; draw lines between cards when you think the concepts are related and add short explanations if you think it is necessary. This is not about doing something right or wrong, but to gain insight into the connections you see between the concepts that have been discussed in this course.”

4.2.3.5 Questionnaire

The questionnaire that had been developed for this study was also used in this small scale study to allow adding, removing or adapting questions for the large scale study. For the design of the (anonymous) questionnaire, the list of ‘constructs and their contrasting poles’ from the study by Verloop (1989, p. 188) was used as a source. Furthermore, the categories found by Holligan (1997), in student responses in a similar study of appreciation of theory, have also been taken into consideration. The fourteen questions relate to the evaluation of the course, and especially to student appreciation of theory as it is expressed in the course (appendix 13). The written response to the questionnaire was set before the interview, to achieve as clear an impression of the students’ opinions as possible. Descriptive statistics of the data (mean and std. deviation; appendix 13) have been determined using the computer software SPSS, version 15.0.

4.2.3.6 Final interview

After the course ended, the students were interviewed individually. This was a semi- structured interview (Kagan, 1990; Fontana & Frey, 2000). The researcher targeted five topics (the lists of concepts, concept mapping, the reflective note, the numeracy test and the questionnaire) and accompanying key questions (see appendix 8). The intention was to gain extra information about the students’ theory use, particularly the character and size of the network of theoretical concepts the students had available. Posing additional questions was determined by the student’s responses; criterion for such use of new questions was the expectation that there was a chance to gain a deeper insight into the meaning the student gave to the concepts and into the quality of the relationships the student made between concepts.

4.2.3.7 Numeracy test

The students did – outside the framework of the course on offer – a numeracy test (appendix 18); the students own numeracy serves as an independent control variable in the study marked by the research questions. A positive correlation is suspected between the ability of the students to solve mathematical problems and their level of use of pedagogical (content) theory. Students who have a high level of skill in solving mathematical problems are functioning at a high cognitive level. That quality is likely also important for using pedagogical content theory at a high level, not least because being able to solve problems in mathematics teaching is a basis condition for one’s

Theory-enriched practical knowledge in mathematics teacher education

functioning in relation to the pedagogics of content. The parallel between the level of one’s numeracy and that of the use of pedagogical theory is likely to be even more prominent in older students than in younger ones, as a result of the growth in experience for older students, particularly where the pedagogical use of theory is involved. The written test contained ten problems and was derived from tests for the subject of mathematics, which were in common use at many Pabos (Teacher training colleges) in the Netherlands at the time (2003) of the numeracy test (Faes, Olofsen & Van den Bergh, 1992; Goffree & Oonk, 2004; Oonk, Van Zanten & Keijzer, 2007). The standard for the test (0-100) was established for the whole group in consultation between the teacher educators and the researcher.