To investigate prospective undergraduate mathematics students’ conjecturing and proving processes, we decided to use computer-supported learning environments that allowed to record students’ verbal face-to-face interactions and their written utterances. All screen and audio activities were recorded by the laptops and transformed into a video file. In general, we assume that the use of “real-time” recordings of students’ interactions during collaborative proof construction activities provides several advantages over other methodologies (such as questionnaires, interviews or real-time observations, etc.) (cf. Roth, 2009):
o Video-recordings as a “real-time” data collection technique allow comprehensive analyses of collaborative conjecturing and proving processes
We assume that video-recordings that capture “real-time” conjecturing and proving processes allow the identification of the process characteristics that may lead to an impasse as well as the process characteristics that may lead to success. “Real-time” conjecturing and proving data
A rating scheme for assessing process characteristics of collaborative conjecturing and proving are also expected to provide information regarding what is going on during collaborative conjecturing and proving processes, how impasses occur, and which activities are promising to overcome them (cf. Savic, 2015a). Moreover, this kind of data may allow to describe the process characteristics that positively influence the generation of conjectures and construction of proofs and the characteristics that predict different patterns of peer collaboration.
o Video-recordings of screen and audio activities allow precise and subtle analyses of collaborative conjecturing and proving processes
As collaborative conjecturing and proving processes are very complex and involve several activities, such as generating examples, applying definitions, or selecting and evaluating the arguments of others, researchers can reach their limits to perceive all aspects that occur simultaneously. In order to avoid missing important details and information that may explain students’ success or troubles in formulating conjectures and generating proofs, video- recordings can be stopped, slowed down and broken down in subsections. Consequently, researchers can code students' collaborative conjecturing and proving processes in multiple passes, and review every interaction the students have made at several times (cf. Roth, 2009).
o Video-recordings of screen and audio activities allow to achieve high inter-rater agreements and the objective coding of collaborative conjecturing and proving processes
Achieving a high inter-rater agreement is a major challenge in the context of assessing the quality of students’ collaborative conjecturing and proving processes. In general, quality judgments require inferences based on the observable data that go beyond counting directly observable events or assigning characteristics to a particular category (cf. Seidel, 2005). A set of video-recordings of previous studies can be used for rater-trainings. Pointing to specific instances occurring in one of the video-recordings can help to establish coding rules and serve as anchor examples. Disagreements can be resolved by explaining one’s coding decision by reviewing the video-recording together. Moreover, a precise coding procedure (see aspects listed under 2) can help increase the likelihood of high levels of inter-rater agreements (cf. Roth, 2009).
o Video-recordings of screen and audio activities allow both qualitative and quantitative analyses of collaborative conjecturing and proving processes
Video-recordings of students’ collaborative conjecturing and proving processes can be qualitatively analysed (e.g., do students’ accurately present their arguments by providing adequate warrants, do they equally contribute to the collaboration process by exchanging and evaluating each other’s idea, etc.). These video recordings can be used to capture multiple qualitative descriptions of collaborative conjecturing and proving processes, but they also allow for more quantitative analysis (e.g., how often do students present new ideas or generate examples, what is the average length of students’ utterances in the collaboration process, how
A rating scheme for assessing process characteristics of collaborative conjecturing and proving many times do students formulate questions, etc.). Another advantage is that video-recordings receive data that can be re-analysed and used for a variety of purposes in the future. For instance, researchers may select some of the video-recordings to provide additional evidence to communicate their results within the mathematics educational community, or to use them as best-practice examples for teaching purposes (cf. Roth, 2009).
Video-recordings initially constitute raw data material. A further decision regarding the coding strategy has to be made. As our research questions address the assessment of the quality of collaborative conjecturing and proving processes, the use of high inference rating scales is reasonable (cf. Clausen, Reusser, & Klieme, 2003; Seidel, 2005). Low inference coding techniques are more suitable for counting how often an event or specific characteristic occurs (e.g., checklists) or for classifying a specific characteristic into one category (e.g., category systems). In general, their coding instructions can be formulated very clearly and achieving a high reliability is much easier than with high inference coding strategies. Due to the fact that instruments such as checklists or category systems often split observational events or characteristics down to the smallest detail, a global view on the underlying construct that should actually be measured is often lost. Based on the experience of previous studies that focussed on process-product correlations within the educational science, the use of high inference coding strategies appeared to be preferable. For instance, the IPN video study has also contributed to strengthen video-recordings as a methodological design and to apply high inference coding techniques (cf. Seidel et al., 2005). The use of global ratings as a particular high inference coding technique enables to capture the content and structures of the underlying construct in a valid way (cf. Clausen et al., 2003; Gartmeier et al., 2015; Newble, 2004). Rating scales produce data that can be handled as approximately interval-level, especially if the endpoints of the scales are considered as the extremes of a continuum (Wirtz & Caspar, 2002). Moreover, the study of Meier et al. (2007) also showed that rating scales provide an adequate technique “to evaluate the quality of collaboration processes on a relatively global level” (p. 71) and that their application is time-efficient, since the transcription of students’ dialogue is not necessarily required. From this perspective and with regard to our research questions and aims, we think that developing high inference rating scales that take the entire collaboration process into account appears to be an adequate coding strategy for analysing video-recordings of prospective undergraduate mathematics students’ collaborative conjecturing and proving processes.