CMU software

During the early 1 9 80s, Anderson and his eo-researchers developed two significant tutoring systems: the Lisp Tutor (Reiser et al. , 1 985) and the Geometry Tutor (Anderson et al. , 1 9 85). One of the main aims was-to j ustify and refine the production system model of learning but, as noted by Anderson, Boyle, Farrell and Reiser ( 1987), they also believed that a model of cognition was essential for constructing any computer tutor. Such a model would have to provide an accurate assessment of the student's current cognitive state, a precise definition of the desired cognitive state and a learning theory to explain how the change from one to the other takes place.

The Lisp Tutor is a program that provides assistance to students as they work on elementary Lisp programming exercises. It has been used for introductory Lisp teaching at CMU since 1 984 and has been successfully marketed (one of the few ITSs to have achieved this status). Evaluation studies (Anderson, 1 990) have shown that students learn more quickly and effectively using the tutor than in self-study.

The expertise in the Lisp Tutor, not surprisingly, is represented by production rules. Each of these rules corresponds to some concept in the language. An example, taken from Corbett, Anderson and Patterson (1990, pp87-88) and transliterated into English, is shown below:

IF the goal is to form a list by inserting newitem at the beginning of an existing list oldlist

THEN code a function call to CONS and set subgoals to code

newitem and oldlist.

The production rule organisation is slightly different from ACT* in that a goal stack is maintained and that a single active goal is popped off the stack during each cycle. This restricts the student to operate in a top-down, left to right fashion. This refined version

of ACT* is called GRAPES (Goal Restricted Production System Architecture). An advantage of this organization in the tutor is that explanations can be related to current goals.

The 325 rules in the tutor are designed with novice learners in mind and do not necessarily correspond to the way that an expert might store the information. Instead, they reflect what protocol analysis has shown to be simple concepts that novices could relate to and remember. As a further variant on ACT*, the original Lisp Tutor stored 475 mal-rules that denote common misconceptions that students may have.

The ideal rules and mal-rules are used for attempting to monitor the student's problem solving procedure. The student is guided along the idealized path represented by the correct production rules (so-called model-tracing). The mal-rules allow the system to check and monitor certain kinds of errors and, as long as the student makes errors that the tutor can recognize, s/he will be allowed to continue. Such deviations from a correct line are strictly controlled, however, since the ACT* model does not cover learning by making mistakes. Consequently, the student is usually quickly brought back into line after an error is detected. Anderson feels strongly that it is unnecessary to attempt to diagnose student misconceptions in detail. He believes 'there is little role for delivering cognitive diagnosis of errors in feedback and in many cases the tutor's feedback should be minimal even to the point of being non-existent' (Anderson, 1 989, p 1 ).

Another feature of ACT* is the use of weak problem solving methods in a problem domain and how it can be used (via knowledge compilation) to produce appropriate domain rules. How this might occur in teaching Lisp is described by Anderson ( 1 987). He shows how both means-end analysis and analogy may be used by students to produce the kind of rule that is embodied in the Lisp Tutor.

The evidence for k nowledge compilation amongst humans was gathered by logging the problem solving times of students using the Lisp Tutor (Anderson, 1 9 87). Analysis shows that students apply rules slowly first time around (presumably using weak methods on declarative data) but rapidly improve thereafter (knowledge compilation). After this initial significant improvement there are smaller gains corresponding to strengthening of production rules.

Improvements in the Lisp Tutor continue to parallel modifications to the ACT* model . For example, a more flexible control mechanism, PUPS (PenUltimate Production S ystem) (Anderson et al. , 1 990), has a more flexible control system than GRAPES so

that a top-down, left-to-right development system is not imposed on the student. PUPS also contains schema-like structures containing special slots for problem solving.

The lower profile but equally ingenious Geometry Tutor (Anderson et al. , 1 985) has been another CMU vehicle for investigating cognitive models. The domain, elementary problem solving in geometry, is very different from that of the Lisp Tutor but, again, the program was found to be successful when used in a teaching situation, in this case tutoring children in Pittsburgh high schools.

As usual, idealized student knowledge is encapsulated in production rules. In keeping with the nature of the domain, both forward and backward inferencing rules are used since it is equally valid to work from the hypothesis or from the premises. Examples of both forward and backward inferencing rules (Anderson et al. , 1 990, p 1 0), are shown below:

IF the goal is to prove triangle XYZ is congruent to triangle UYW and X, Y, W are collinear

and U, Y, Z are collinear

THEN conclude angle XYZ = angle UYW because of vertical angles IF the goal is to prove triangle XYZ is congruent to triangle UVW

and XY=UV and YZ=VW

THEN set a subgoal to prove angle XYZ = angle UVW so SAS can be used.

An i nteresting feature of later versions of this program, as noted in Section 2.5.5, is the clever interface. Not only does it make the package easy to use but also reduces the student's cognitive load by providing a visual analogue of the proof procedure, a technique that is sometimes called rei.fication (Wenger, 1 987). It is this aspect of tutoring in geometry that has been the focus of recent research in the area (Koedinger, 1 99 1 ; Koedinger and Anderson, 1 993).

The visual representation of the proof graph and the automatic linking of logically valid statements to the subgraphs already built up provide positive guidance to the student. However, as yet, the tutor does not contain mal-rules and does not provide feedback on logical errors. Hints can be given when the student has gone wrong but these are not dependent upon the kind of mistake the user has made.

Other, more recent, projects at CMU include the development of a tutor for high school algebra (Lewis et al. , 1 990) and the extension of the Lisp Tutor system to include facilities for teaching Pascal and Prolog (Anderson et al., 1 992). In this last paper, some principles for a general architecture for tutoring systems are proposed, paving the way for the development of an intelligent authoring system.