Evaluating and Categorising Student’s Behaviour

Upon completing each tutorial, it is useful to assign a stereotype according to the student’s understanding to allow tutors to assess the student further. Information required for the stereotyping is obtained from the student’s behaviour as he/she interacts with the ITS. Sison and Shimura (1996) defined student’s behaviour as any observable response that is used as input to the student modelling process. In the C++ STL ITS, various information measured during the interactions is evaluated, and subsequently used to categorise the student. The pre-test performance results include the number of correct and incorrect answers, and conditional probabilities of understanding for each topic. As the student attempts the tutorial, the time spent to complete the tutorial, number of attempts to answer correctly, number of hints selected and the updated conditional probabilities are measured. The information observed is employed to categorise the students into four common stereotypes that concern the knowledge of student: novice, beginner, intermediate and advanced.

The Bayesian approach efficiently calculates the conditional probabilities of student’s understanding. However, it is not capable of dealing with uncertainties in categorizing the student’s understanding given the variables like tutorial duration, number of attempts and number of hints selected.

The proposal is to apply fuzzy logic techniques to provide human-like diagnosis of the student’s knowledge. A general fuzzy expert system based on the application of fuzzy set theory will be implemented to perform the approximate reasoning. The

technique proposed to build fuzzy expert system is the Sugeno method. The method was named after Michio Sugeno (Sugeno, 1985). The operation of the fuzzy expert system involves the execution of four major tasks: fuzzification, rule evaluation, aggregation and defuzzification (see Figure 4.6)

Figure 4.6 The Four Tasks in the Fuzzy Expert System

Fuzzification involves the choice of parameters, fuzzy input and output variables and defuzzified output variable(s), definition of membership functions for the input parameters and the description of fuzzy rules. The main purpose is to convert crisp values into a fuzzy set. The membership functions defined on the input parameters are applied to their actual values to determine the degree of membership for each fuzzy rule.

A fuzzy rule is an IF .. THEN rule of the standard form: IF x is A THEN y is B

where

 x and y are linguistic variables

 A and B are linguistic values determined by fuzzy sets.

Fuzzification Rule

Evaluation

Defuzzification Aggregation

The x is A part is known as the antecedent or premise, whereas y is B is called the consequent or conclusion. In the Sugeno-style fuzzy inference, a singleton is used as the membership function of the rule consequent. A singleton is a fuzzy set with a membership function that is unity at a single particular point on the universe of discourse and zero everywhere else (Negnevitsky, 2005). The most commonly used zero-order Sugeno fuzzy model applies a constant value to the linguistic value (i.e. B) of the consequent.

Fuzzy sets can have a variety of shapes represented by various membership function. A regular membership function will be applied to the C++ STL ITS fuzzy model. In most cases, a triangle or a trapezoid is adequate to represent the expert’s knowledge.

To categorise the students, the C++ STL ITS will adopt the following input as the linguistic variables with their corresponding linguistic values:

 Conditional probabilities - high, medium and low

 Time to complete a tutorial - short, medium and long

 Number of attempts - high, medium and low

 Number of hints selected - high, medium and low

 Understanding - novice, beginner, intermediate and advanced

The linguistic variable Understanding is the desired consequent, where as, the rest are the antecedent.

Subsequently, the student’s understanding is captured in fuzzy rules: Examples:

Rule 1:

IF conditional_probability is high AND time_taken is short

AND number_of_attempts is low AND hints_selected is low THEN understanding is advanced

Rule 2:

IF conditional_probability is low AND time_taken is very long

AND number_of_attempts is very high AND hints_selected is very high THEN understanding is novice

Rule Evaluation: With the definition of the rules, membership function and fuzzified input, the truth value for the antecedent of each rule is computed, and applied to the consequent part of each rule. The output is one fuzzy set assigned to each output parameter for each rule. The max_min operator will be applied to evaluate the rules (Hopgood, 2001).

In the Aggregation process, all the fuzzy sets assigned to each output variable are combined together to form a single fuzzy set for each output variable. The membership functions of all rule consequents from the previous stage are combined into a single fuzzy set.

Defuzzification is the process of converting fuzzy output set into the desired crisp value. The input for the defuzzification process is the aggregate output fuzzy set and the output is a single crisp number. The output is simply calculated by finding the weighted average of the singletons obtained from the aggregation operation.

Another method for fuzzy inference is known as the Mamdani method. It was introduced by Professor Ebrahim Mamdani of London University who built one of the first fuzzy systems to control a stream engine and boiler combination (Mamdani & Assilian, 1975). This method uses the centroid technique for defuzzification which relies on using the centre of gravity of the membership function to calculate the crisp value of the output parameter. Although the Mamdani method is widely accepted, its defuzzification process is computationally intensive. On the other hand, the Sugeno method is computationally efficient, works well with optimisation and adaptive techniques (Negnevitsky, 2005). The singleton output functions are able to satisfy the requirement of C++ STL ITS to categorise the student. This makes the Sugeno method an attractive choice for this ITS.