Adaptable Feedback

Feedback follows after instruction and can have major influences on learning (Hattie & Timperley, 2007). Feedback is information provided by an agent such as by a computer- based learning environment or by a learner him or herself. That is, feedback can come from an external source or internally from a learner (ibid). Feedback in an instructional setting is considered to be all information provided after a learner has responded to a stimuli that informs a learner about his or her actual state of performance (Narciss, 2013).

Chapter 3: Instructional Support for the Acquisition Diagnostic Competence

For feedback reception metacognitive skills are crucial (Narciss, 2008). Metacognition is knowledge and monitoring of one’s own cognitive processes (Flavell, 1979). Feedback aims at reducing the discrepancy between a current and a desired state (Hattie & Timperley, 2007).

Feedback can help to detect errors or knowledge gaps, and give strategically useful information (Narciss et al., 2014). Feedback in instructional contexts can sometimes not clearly be distinguished from instructional explanation (Hattie & Timperley, 2007), as also instructional explanation should be relevant for the misunderstandings of a learner to foster elaboration (Webb & Mastergeorge, 2003). An example for the difficulty to distinguish feedback and instructional explanation is revising instructional explanation. Whereas standard instructional explanation provides learners with basic understanding of a topic, revising instructional explanation targets gaps and flaws in already gained knowledge (Wittwer & Renkl, 2008).

Effective feedback relates to three questions and also to dimensions of learning. The questions to be answered by feedback are (1) What progress is being made toward the goal? (2) What activities need to be undertaken to make better progress? and (3) What are the goals? (Hattie & Timperley, 2007). The dimensions of learning involve task performance, understanding of a task, metacognitive processes, and self. Related to diagnostic competence, the first and second questions correspond to strategic knowledge, as they involve problem-solving strategies and heuristics in relation to a specific case. The third question corresponds to conditional knowledge, as it is about the goals of a procedure and of its rationale. Therefore, to foster diagnostic competence, it might be beneficial to structure feedback with regard to these dimensions.

To make the erroneous worked examples promising for learners with low prior knowledge in addition to learners with high prior knowledge, feedback in which the error is explained and linked to the theoretical background could be important. However, instructional explanation failed to improve learning in several studies.

Instructional explanation can be beneficial in helping students apply their existing knowledge to new cases and also can fill gaps in knowledge (Wittwer & Renkl, 2008). Even though a meta-analysis showed that providing instructional explanation had a positive effect on conceptual knowledge, a negative effect on problem solving skills in math and no effect in science or learning science (Wittwer & Renkl, 2010) were found. Compared to worked examples with prompts to self-explain they were not beneficial. In three experiments on electrical circuits that used worked examples, withholding instructional explanation was beneficial (Richey & Nokes-Malach, 2013). Richey and Nokes-Malach (2013) have hypothesized that instructional explanation may discourage constructive behaviors.

Chapter 3: Instructional Support for the Acquisition Diagnostic Competence

Revising instructional explanation that targets gaps and flaws (Wittwer & Renkl, 2008), and is very much comparable to feedback, failed to enhance learning in several studies (Chi, Siler, Jeong, Yamauchi, & Hausmann, 2001; Schworm & Renkl, 2006). Sánchez and García-Rodicio (2013) note that instructional explanation has not been marked as corresponding to the learners’ misunderstandings in the previously mentioned studies. Thus, the additional information may have been experienced as redundant to the learning material. Their own studies show an advantage of explicitly marking instructional explanation as corresponding to errors or misconceptions of leaners (ibid).

There could be various reasons why instructional explanation and feedback failed to be beneficial for learning, such as the prior knowledge of the learners. A study on how to foster diagnostic competence in medicine (Stark et al., 2011) and other studies (e.g. Strijbos, Narciss, & Dünnebier, 2010) showed that elaborated feedback is not beneficial for every learner. Instructional explanation of the rational of a procedure is valuable for learning in the beginning: however, it can become redundant during learning and should be faded out after some time (van Gog et al., 2008). The redundancy of the explanations could cause an expertise reversal effect (ibid), as with more expertise learning may even be hampered by additional explanations (Kalyuga et al., 2003). That these unnecessary explanations and redundancy can also be detrimental for learning is also supported by other authors (Kalyuga & Renkl, 2010). A hint in that direction could be that feedback with a fixed format given after self-explanation prompts had negative effects (Gerjets et al., 2006). An possible reason for this could be that the instructional explanations were not well adapted to the prior knowledge of learners (Wittwer & Renkl, 2010), and they may not been given at the time a learners needed them (Renkl, 2002).

Feedback given to learners if they are at an impasse and cannot self-explain on their own seems especially helpful (Renkl, 1997; Stark, Gruber, Mandl, & Hinkofer, 2001; Stark, 1999). But learners, even if they are formally at the same educational level, may differ substantially with respect to prior knowledge. An automated adaptive feedback that is specifically tailored to the needs of the individual learners would be the best solution. To be adaptive, a tutor has to monitor the understanding of the learner (Chi, Siler, & Jeong, 2004). Even human tutors with high conceptual understanding of the content domain fail to diagnose students’ false beliefs and knowledge deficits accurately and accordingly, have difficulties adapting their instructional explanation to the learners’ needs (Chi et al., 2004).

To give adaptive feedback after an error would required knowing exactly what the error was in order to decide on the adequate instructional support (Aleven, Stahl, Schworm, F. Fischer, & Wallace, 2003). But in a complex field such as in education or in medicine, the generation of the knowledge base that would be needed to analyze the learners’ understanding automatically is currently out of reach (M. Fischer et al., 2008).

Chapter 3: Instructional Support for the Acquisition Diagnostic Competence

Accordingly, it is difficult to adapt instructional explanation to the needs of the learners automatically, particularly in complex domains (Aleven et al., 2003).

A possibility for adaptive feedback would be to let learners choose their self- explanations from a set of multiple-choice questions. This procedure was effective in some studies (Atkinson et al., 2003; Conati & VanLehn, 2000) but had no effect in others (Gerjets, Scheiter, & Schuh, 2005). However, such a procedure is only of limited use, as a wrongly chosen self-explanation only contains limited information about the misconceptions a learner might have.

A possible way to implement some adaptability is to let learners decide on the extent of feedback they need (Leutner, 2002). Help on demand in combination with self- explanation prompts and worked examples can be beneficial for learning (Renkl, 2002). On-demand help is help that the learner actively requests e.g. by clicking on a hyperlink (Aleven et al., 2003). Instructional explanation on demand can benefit learners with low prior knowledge without harming the learning of those learners with high prior knowledge (Renkl, 2002). Through letting learners decide about the level of detail of the feedback, the autonomy of the learner is fostered and therefore the conditions for intrinsic motivation are improved (Deci & Ryan, 1993). Learner control in computer-based environments can increase interest and motivation and may also help the learner to adapt the learning environment to his or her cognitive needs (Scheiter & Gerjets, 2007). In addition, to let learners decide on the level of feedback by clicking on a link is an active activity as learner is physically doing something (Chi, 2009). Being active may activate existing knowledge so that new knowledge can be added easier (ibid). Having learners decide on the help they need may also provide them the opportunity to find their own explanations (Anderson, 1993).

If feedback is structured with regard to the previously described types of knowledge the learner may also have the opportunity to focus on the knowledge he or she needs. For example, after deliberately relating prior knowledge to case information, a learner might recognize a wrong procedure (question 1), but it could still be the case that he or she does not know how to proceed (question 2) or what the goal of procedure is (question 3). With an adaptable feedback method the learner would not have to scan through all the information, but could decide upfront if further explanations about a certain type of diagnostic knowledge are necessary.

Learner control poses high demands on the learner (Scheiter & Gerjets, 2007). The effectiveness adaptability, that is to let learners decide on, e.g. the level of feedback they need, requires a certain level of metacognitive competence, which is missing in some learners (Stark & Mandl, 2002; Stark et al., 2008). Learner-controlled adaption in which the learner actively chooses instructional activities can be beneficial, but learners often

Chapter 3: Instructional Support for the Acquisition Diagnostic Competence

lack the metacognitive ability to decide on the most beneficial activity for learning (Narciss, 2008). For example, it was shown that feedback on demand was not used very often (Corbett & Anderson, 2001). Adaptability, accordingly, can also be problematic because learners with low prior knowledge are often bad help seekers (Aleven et al., 2003). In computer-based learning environments help seeking consists of five steps. The steps are (1) becoming aware of the need for help. Self-monitoring skills are crucial (Aleven et al., 2003). (2) The decision to seek help, which may be less influenced by help-seeking costs in computer supported leaning, e.g. by the risk of being seen as incompetent. To let a learner simply click on a link to ask for help might further reduce help seeking costs. (3) The identification of a source for help, which in case of an adaptable feedback measurement is very easy. (4) Making use of provided help. In computer-based learning environments the help may not always be tailored at students needs. Accordingly, the learner needs to filter the information provided and judge the usefulness for the problem at hand. (5) Learners evaluate the help-seeking process (Aleven et al., 2003). Help-seeking activities are not easy processes and from the five steps it gets obvious why help seeking can increase cognitive load (Aleven et al., 2003). Even though in a computer-based learning environment that provides adaptable feedback that might be less important as, for example, the step two and three are less demanding than in other setting such as in a classroom. Learners tend to overestimate their understanding (Chi et al., 1994) and thus refrain from seeking help in the first place. It is possible that learners who need additional explanations the most, are the least prone to ask for them, in some cases because they do not even know they need it (Gräsel, F. Fischer, & Mandl, 2001; Narciss, Proske, & Koerndle, 2007).

One of the reasons why the combination of adaptable feedback with self-explanation prompts is promising is that the combination of both might help learners to realize their need for additional explanation. In the next section the prospects of a combination of self- explanation prompts and adaptable feedback are analyzed.