Adaptable Scaffolding

The strategy used for PAS is critical in CBL systems, since there is a potential for the system’s trustworthiness to become questionable and, eventually, the user may abandon the learning process. During the scaffolding process the system frequently measures the learner’s performance. It could be expected that a learner with a strong mental state will answer an easy question correctly and, similarly, a learner with a weak mental state will answer a difficult question incorrectly. However, this is not a certainty, since a lucky guess or careless mistake may still play a role. A gifted learner may be annoyed if detailed feedback is given for just a careless mistake. Similarly, a weak learner may be frustrated if the system asks the learner to try again to solve a problem to which they have not a clue to the answer and it may well be beyond their Zone of Proximal Development.

To incorporate adaptability in the scaffolding process, as previously mentioned, the Learner model in LOZ retains some numerical measures, including the Strength of Mental States (SMS). Earlier, a simple method was described to determine the required level of PAS option, based on the variables P and SMS. The ranges of the variable SMS, specified in the look-up table (Table 6.1), are discrete. The boundaries do not overlap. In probability terms, the variable SMS is treated as a binary variable, which means SMS is

true if the learner’s strength in the current mental state is ‘sufficiently strong’ to perform a certain task, since otherwise it would be false. However, this interpretation is not solid as the phrase ‘sufficiently strong’ is rather vague. Corbett et al (1993) keep a similar variable

to aid knowledge tracing, which takes one of two discrete values: learned or un-learned.

Alternatively, instead of a single state ‘sufficiently strong’, different linguistic terms such as ‘strong’, ‘medium’ and ‘weak’ may be used for SMS to describe different potential states. A binary variable (true or false) may be associated with each state to denote the corresponding system’s belief. In a similar approach, Mislevy et al. (1996) in HYDRIVE

use linguistic terms, such as ‘expert’ and ‘good’ to represent different knowledge levels, and they use BN for Learner modelling. Though these states look like fuzzy ones, in actual fact they are not. According to their definition, an ‘expert’ cannot be ‘good’ at the same time. In contrast, during real life situations such as learning and mentoring, these clear-cut distinctions cannot be made for variables which may include the knowledge level of a learner and/or the difficulty level of a question (this will be discussed in detail later in this chapter).

Naturally, the variable SMS, which represent the potency of knowledge, cannot take concrete yes-or-no types of values and, therefore, should be treated as a fuzzy variable. Hawkes et al state, “The use of fuzzy terms allows for imprecision and vagueness in the values. This provides flexible and realistic representation that easily captures the way in which the human tutor might evaluate a student” (Hawkes et al. 1990, p. 416). In a

simple case, the variable SMS may take two linguistic values with overlapping ranges: ‘sufficient’ (say 40-100) and ‘not sufficient’ (say 0-60). The strength of a student, with a SMS between the values 40 and 60, will be both ‘sufficient’ and ‘not sufficient’ to a certain degree. In this research, the SMS takes three linguistic values: strong, medium and weak. For a given SMS, the degree of membership for these sets will be determined by a membership function (Figure 6.2). For example, a score SMS=75 means that the system believes the learner is at the 75% strong and 25% medium levels, related to the current mental state. By convention, this will be denoted as

µ

µ

In this research, unlike the other CBL systems that use fuzzy logic for Learner modelling, a definite membership functions is used. Katz et al (1994) use five overlapping sets to

model the variable ‘knowledge state’ of a student. Hawkes et al. (1990) use seven

intervals to model a variety of variables, from motivational state to error count. Each interval is assigned a number ranging from one to seven. Neither research group employs a single crisp value to represent the ‘knowledge state’.

After an MC test is answered, the performance level P is measured. For a traditional MC test, P is just a binary variable and takes two concrete values: correct or incorrect. As previously discussed, if the confidence-based MC tests are used, P can be a fuzzy variable and may take three linguistic values: Correct, Incorrect, and Medium (Figure 5.8). In Section 5.3, different levels of PAS options were designed for each type of performance (Tables 5.1, 5.2, and 5.3). Similar to SMS, it is natural that the boundaries of different levels of PAS options cannot be defined strictly. Basically, the PAS levels are different linguistic values related to the possible pedagogical actions. Therefore, the PAS levels for each type of performance can be treated as a fuzzy variable. The PAS variables for Correct, Incorrect and Medium performances are named PASc, PASw and PASm, respectively. The different levels of the variable PASc will be, in their increasing levels, denoted by PAScL1 to PAScL5 (similarly, the notations PASwL1 to PASwL5 and PASmL1 to PASmL5 will be used for PASw and PASm, respectively). The fuzzy set membership function in Figure 6.3 is given for the variable PASc. The same function will also be used for the other two PAS variables. For example, PASc=87.5 means the required level of PAS option is both, PAScL5 and PAScL4 at 50%.

Figure 6.2 Fuzzy Membership Functions for SMS

medium strong weak 0 20 40 60 75 80 100 75 -- 25 -- 100 -- 50 -- SMS ip (%)

Furthermore, in the previous technique, PAS levels are selected based on the variables SMS and P only (Table 6.1). In addition to SMS and P, the difficulty level of the question also plays an important role in determining suitable pedagogical requirements. The domain expert may initially propose the difficulty level for each question. Naturally, the difficulty level for an MC test will closely correlate with the related scaffolding stage. Some questions are considered difficult and some others are considered easy. However, this division is usually vague − clear-cut boundaries cannot be assigned. Therefore, another fuzzy variable, D, which takes three linguistic values: ‘hard’, ‘moderate’ and ‘easy’, is assigned to represent the difficulty level of an MC test. The same fuzzy membership function given in Figure 6.2 may also be used for D.

Apparently, the variable P depends on SMS and D. However, once P becomes available, the suitable pedagogical action depends on all three variables. In other words, for a particular learner, after answering an MC test, the required level of PAS option could be determined by the values SMS, P and D. These causal relationships are represented by the directed acyclic graph in Figure 6.4. The fuzzy rules that govern these casual relationships are given in Table 6.2. It may be noted that the fuzzy rules closely reflect the decision making processes of a typical human mentor in different learning situations. For example, human mentors usually expect that the experts will perform well on easy assessments. Therefore, when an expert correctly answers an easy question, mentors will probably just confirm this answer and then move onto the next section (or offer some more difficult questions). In the meantime, if an expert fails in such a situation, the mentors will usually treat it as a slip-up, and give the learner another chance to correct their error. The first row in the given fuzzy matrix (Table 6.2) reflects the above situation. It is notable that the variable P is considered binary (it may be fuzzy or non-fuzzy) in Table 6.2. Another value

Figure 6.3 Fuzzy Membership Functions for PAS

PAScL5 PAScL4 PAScL3 PAScL2 PAScL1 10 12.5 25 50 75 87.5 100 100% 50% | | | | | | | | | | PAS De g ree of Membersh ip (% )

table.

As previously stated, after a student has learned the material related to a sub-concept, LOZ assumes that they are in the corresponding basic mental state. In other words, this will be the first scaffolding stage related to the current sub-concept. To ensure the validity of this assumption, the system will give a relevant MC test. Depending on factors such as performance in the test and the difficulty level and the current strength of mental state, the system will select the appropriate level of PAS option (which provides appropriate feedback, and then selects the next suitable scaffolding stage). As previously assumed, SMS may be 50 (medium-100%) for the first basic mental state of a fresh main-concept. The effect of uncertainty is inevitable in the factors SMS and P. To lessen the effect of

Mental State (SMS)

Difficulty

level (D) P =Incorrect PASw value P = CorrectPASc value

Strong Low PASwL1 PAScL1 Strong Moderate PASwL2 PAScL2 Strong High PASwL3 PAScL3 Medium Low PASwL2 PAScL2

Medium Moderate PASwL3 PAScL3

Medium High PASwL4 PAScL4 Weak Low PASwL3 PAScL3 Weak Moderate PASwL4 PAScL4 Weak High PASwL5 PAScL5

Table 6.2 Fuzzy Rules for PAS

SMS

P

D

PAS

uncertainty, in selecting appropriate pedagogical action, a well-defined fuzzy mechanism is incorporated in LOZ. The scaffolding process is now adaptable to individual learners.

In summary, since the critical response type tests are not used in the first phase, interactive problem solving support is not possible. Therefore, diagnosis at the micro- level is not necessary; however, macro-level diagnosis is performed at this time. The distracters for the MC tests are selected to match the potential misconceptions. Neither simple heuristics (as in the traditional CBL systems) nor a complex utility function (as in systems such as CAPIT (Mayo et al. 2001)) is used in this model. Instead, some natural

fuzzy rules, which are almost parallel to the human tutor’s decision making processes, are used. A well-defined fuzzy rule application process is designed to reduce the impact of uncertainty in the pedagogical action selection process. Nevertheless, whilst measuring performance, the time taken to answer a question is not taken into consideration. In reality, it can be expected that a good student will answer a question quickly (and correctly). The underlying fuzzy mechanism for PAS, used in this research, will be illustrated in the next section.