Four main issues raised in this study call for further research.
a) Although performance tests are not the only way to measure learning, it re- mains a legitimate and largely-practised way to assess mastery of course con- tent. In this perspective, final scores should reasonably be expected to reflect benefits resulting from the RAs. It has not happened here, at least in comparison with the control group. This lack of benefits from note-taking contrasts with other studies in the field (Hwang, Wang, & Sharples, 2007; Nokelainen, Miettinen, Kurhila, Floréen, & Tirri, 2005). Further empirical studies can help to sort out what the effects of annotations “ought to be” from what they actually accomplish, and most importantly, in what instructional context.
b) Besides considerations tied to exam improvement, the functional complexity of note-taking deserves further investigation. For instance, consolidation of re- sults is needed regarding how embedded, frequent and structured stop-and-think beacons, like the annotations, influence the perception of the learning experi- ence (sense of control, feeling of learning, narrative account). More research needs to be undertaken to see if such prompts for reflective appraisal of the study material can be related to ownership of learning and sharper feelings of personalisation. In such an approach, personalised learning might be seen as a consequence of acting and reflecting and not as the result of a decision taken by an external agent like a teacher or an adaptive system (Verpoorten, 2009; Verpoorten, Glahn, Kravcik, Ternier, & Specht, 2009).
c) On a more fundamental level, the study findings, and especially those related to the effects of combined reflective enactment rates (annotations, revision of annotations, page re-visits), highlight the intellectual dynamics at work in deep approach of study material. Similarly, when they recommend to teachers to con- sider incorporating various explicit and convergent activities (making annota- tions, reviewing them) that can foster meta-cognitive development into the learning process, Hwang, Chen, Shadiev, and Li (2011) also hit the interplay of several reflective behaviours. Further work is required to establish whether it could be a characteristic of high achievers and a hallmark of intellectual life in general to operate an “active study”, defined as an ongoing crisscrossing, a pe- riodic and persistent to-and-fro mental move, between a primary learning activ- ity and secondary reflective or meta-cognitive enactments. Research in self- regulated learning points in that direction by underlining the mastery of meta- cognitive activities by high performing students. However, practical ways to evidence and sustain this interplay between cognitive and meta-cognitive land- scapes must also be envisaged from an instructional design viewpoint.
d) In this study, the relationship between the average mark at the test and the annotation activity greatly varies according to how the note-taking activity is contextualised. When the mere fact of taking frequent notes is considered, the mean score at the test for the note-takers is between 6 and 6.4 (see Table 5.1). When this amount of annotation is related to the group average (contextualisa- tion 1), the mean score for the above average note-takers is 6.6 (see section “Amount of reflective enactments and mark at the test”). When the annotation rate is linked to the browsing rate, the mean score jumps at 7.1 (contextualisa- tion 2. See the section “Isolated/combined reflection rates and mark at the test”). When the annotation rate is coupled with the browsing and the visit to the dashboard rates (contextualisation 3), the mean score can achieve 7.8 (same sec- tion). An important issue for future research on annotations is also tied to the selection of relevant evaluation approaches.
Annotations as reflection amplifiers | 93
Conclusion
A growing literature extols the importance to instil reflection and deep ap- proaches to learning in tuition. However, practical and systematic ways to oper- ate are not conspicuous, at least when it comes to reflection in methods of learn- ing considered as traditional or transmissive (Terhart, 2003), in contrast to con- structivist methods (problem-based learning, collaborative learning) wherein reflection is claimed to be “built-in” (Hmelo-Silver, Duncan, & Chinn, 2007; Kirschner, Sweller, & Clark, 2006; Sweller, Kirschner, & Clark, 2007). This chapter inquired the question: how to induce a more thoughtful autonomous study of learning material? To answer, the experimental setting artificially in- creased the number of annotations, conceptualised as frequent tinglings for re- flection while reading and purposed to support a persistent dynamic mental en- gagement with the reading material. An assumption guided this work: that such a kind of active and reflective posture to learning, which constantly articulates the cognitive and the meta-cognitive landscapes, is a key feature of intellectual life. The experimental setting presented here was a simplified attempt to mimic and externalize such fundamental inner dynamic processes via an annotation tool. Eventually, annotations taken alone did not really measure up. However, their combination to other forms of engagement with the material provided evi- dence that the insertion of affordances to amplify reflection is worth considering in connection with cognitive and meta-cognitive learning benefits.
Reflection amplifiers – A systematic research review | 95
Chapter 6
Study 5 (collateral): Reflection-in-action
prompts – A systematic literature review
Chapter 6
Reflection amplifiers – A systematic research review
Abstract
This chapter presents a systematic review of the research conducted in the field of reflection prompts in technology-enhanced learning. The review ultimately leads to the identification of 29 empirical experiments whose main concern is to infuse reflection-in-action affordances within a primary learning task. This state-of-the-art report inspects the theoretical background backing up the reflection prompts, exhibits their instructional settings, categorises their interaction patterns and modalities, synthesises their effects and analyses their technological foundations. Drawing on the review’s findings, directions for future work are documented.
This chapter is based on: Verpoorten, D., Westera, W., & Specht, M. (2012c). Reflection amplifiers in technology-enhanced learning – A systematic research review. Manuscript submitted for publication.
“It requires gentle but firm direction to transform personal reflection into academic outputs.” (Fielden, 2005, p. 469)
This chapter presents a systematic literature review on reflection in Technology- Enhanced Learning (TEL). This effort is relevant because reflection is generally assumed to be among the strongest influential factors of learning. The lens of the review is put on reflection-in-action prompts because, in contrast with tech- niques stimulating after-the-fact contemplation of learning situations (e.g., port- folios), they have so far received limited attention from researchers and practi- tioners.