Timing of Positive Feedback

In general, tutoring systems vary on their policies about when to give feedback. Scheduling policies typically fall under three broad categories, immediate feedback, delayed feedback and demand feedback. Immediate feedback as observed in ANDES, Algebra Cognitive Tutor [Gertner and VanLehn 2000] and AutoTutor [A.C. Graesser and TRG 2001], give feedback immediately after every student step. Sherlock [Katz, Lesgold, Peters, Eggan and Gordin 1998] gives feedback mostly after the student has finished troubleshooting, that is delayed feedback and immediate feedback only if the student step violates a safety regulation. In SQL-Tutor feedback is given only when the student clicks on the Submit button, in other words when the student demands feedback. More complex policies are possible where the policy is a function of the student’s com- petence. Consider for example a student who is approaching mastery of a given domain. It would make sense to provide less feedback to such a student effectively delaying the feedback as compared to a less competent student who might need a more immediate feedback and prompt feedback intervention scheme.

The above feedback policies have been considered in formulating the scheduling of positive feedback. In one-to-one human tutoring, the student typically makes an attempt at the problem before the tutor has any intervention, at which time it is usually based on some error that the tutor has identified in the student’s solution. The student continues

to make such attempts until the problem is solved, effectively submitting each attempt for the attention of the tutor who provides feedback. We propose that positive feedback be given on a demand basis as is currently facilitated by SQL-Tutor and to mimic the actions of human tutoring. However, the system should not give positive feedback on each correctly used constraint, as the amount of such feedback would be overwhelming. Instead, we identified events in the student’s tutoring experience which should trigger positive feedback. It is at these points that we hypothesize positive feedback is most effective. The positive feedback provided to a particular student will depend not only on the submitted solution, but also on the student’s knowledge (as captured by the student model) and the state of interaction. We developed four general cases when positive feedback should be given:

• When the student is expressing uncertainty but nevertheless does the right thing • When the student is too paralyzed to do anything at all and requests assistance. • When the student has overcome aspects of the domain commonly agreed upon as

being difficult and challenging

• When the problem or task has been successfully completed and at other major goals within the tutoring session

Giving student feedback under uncertainty

In this section we consider giving positive feedback under uncertainty that is, when the student is expressing uncertainty but nevertheless does the right thing. Consider a situation where a student is uncertain or simply does not know the correct format for the date within SQL. If the student gets the format wrong, the first time or a number of times after that, then we are certain of the student’s knowledge of SQL date formats. It is either they are uncertain of the format, maybe toggling between two options or simply do not know and is guessing.

In either case the student has submitted an incorrect answer and based on our hy- pothesis, if the student is uncertain, that is, choosing from several chunks of knowledge or problem strategies trying to find the correct one or simply guessing (scientific guess- ing), positive feedback will help reinforce those knowledge chunks which led to the student obtaining the correct answer. In one-on-one human tutoring uncertainty is typ- ically dealt with through probing and providing hints. Tutors typically use scaffolding techniques to break the problem into simpler components isolating the knowledge com- ponent where the uncertainty occurs, encouraging the student to make an attempt and providing hints and suggestions until the student has mastered the concept. Correct responses during this period of learning are often followed by a series positive feed- back messages geared at reinforcing these newly learnt or successfully applied domain principles.

Giving positive feedback under paralysis

During tutoring there will be moments where the student is paralyzed, that is, totally and absolutely unsure of the next step. In such situations the student is unable to proceed without additional aid and feedback comes into play. Sometimes however low-level negative feedback is not sufficient, particularly when the student is lacking some incite knowledge critical to solving the problem. Therefore despite tutorial negative feedback the student does not obtain the correct solution and even when negative feedback is given, the information provided does not seem to assist the student in any way and the student is no better off than previous. Continuous attempts at obtaining correct solutions result in violated domain principles.

In other cases the student might be oblivious as to the next step or to the solution path having no clue or inclination as to what is a correct strategy to solve the problem. In this case the student is too paralyzed to do anything and can only move forward through the assistance of the tutor in the form of some more direct hint on what the solution should be.

In either case, if more specific but indirect feedback is given, that is, the student is given an hint to the answer but not the answer itself, and the student successfully uses this added knowledge to correctly infer the answer, then positive feedback should be given. It shows that the student had at least some knowledge of the problem re- quirements which they successfully applied. Based on our hypothesis positive feedback should increase student confidence. It will also decrease uncertainty over the student’s existing knowledge, decrease uncertainty over the knowledge given within the hint state- ment which was once lacking and strengthen the connection between these two pieces of knowledge chunks.

Giving positive feedback under certainty

There will be instances when the student is absolutely certain of their actions whether as a result of prior knowledge or learnt through the system via error correction in prior problem steps. We have identified two instances under certainty when feedback should be given.

Giving positive feedback at difficult and challenging domain concepts Certain parts of the domain are difficult and challenging increasing cognitive load and the amount of active information processing required from the student. This requires not only domain knowledge (e.g. facts, concepts, events, rules, theories and models) and task specific procedural knowledge (e.g. sorting and drawing) but also the development of more specific and general meta-cognitive skills. In this research we propose giving positive feedback when the student correctly and adequately deals with such situations. Of course this will vary from domain to domain. This will require cognitive task analysis to identify domain specific knowledge, the cognitive skills (e.g. classification, inference and memory) related to these items and the expert skills required to master these tasks.

Giving positive feedback at major goals within the tutoring session When the stu- dent gets the entire problem correct then that represents a significant achievement. If

a student for example, gets a very difficult and complex problem correct on their first attempt or satisfies a difficult constraint the first time it is relevant, then it potentially signifies that the student has mastered those aspects of the domain addressed by that particular problem or constraint. In this case we propose giving positive feedback to in- dicate to the student the magnitude of their accomplishment and to reinforce that correct response.