Predictive Decision-Making

The developed algorithm for skill selection is comparable to the vocabulary learning technique called spaced repetition, which is implemented, for instance, in the Leitner system (Leitner,1972, p. 64ff.). It enables the system to choose the next skill to address that maximizes the beliefs of all skills while bal- ancing the single skill beliefs among each other. To achieve this, skills with lower beliefs are prioritized and repeated more frequently while well known skills are addressed with a lower frequency (spaced rep- etition). This behavior can be modeled easily by calculating the Kullback-Leibner divergence (KLD) between the current and the desired skill beliefp(S_opt). Since the application of the KLD requires at least some probability mass in each bin,p(Sopt)is defined by containing≈99.999% of the probability mass in the last bin, meaning, being≈99.999% sure that the learner has mastered a skill to100%.

next_skill=argmin

St i∈S

[

α(S_it)·KLD(p(S_it),p(Sopt))] (5.8) Srepresents the set of all addressable skillsSithat can be taught. As mentioned above, in the scope of language learning this set includes a huge number of skills/words and, thus, should be split into smaller chunks, so that the skill selection algorithm has to evaluate just a small part of the possible skill space and remains tractable. To regulate the skill occurrence frequency and to achieve the proposed spaced repetitioneffect of maximizing, as well as balancing all skills, the factorα(S_it)is added. It ranges from0.0to1.0and is decreased by0.3each time a specific skillSiis addressed and is increased again by

5.2. Adaptive Bayesian Knowledge Tracing (A-BKT)

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

Skill belief of Sti 0.0 0.2 0.4 0.6 0.8 1.0 (S t i,a l )

easy task medium task hard task

Figure 5.3: Gaussian distributions with respect to each task difficulty used forβ(S_it,a_l).

0.15if another skill is being practiced. This prevents the system from focusing on just one specific skill by addressing it continuously until a sufficient number of correct answers is provided, the skill belief is high enough and in balance with the remaining skills again. However, this results in a slightly different behavior as in the traditionalspaced repetitionsystem.

In thespaced repetitionsystem all words are practiced at least once until the “spaced” repetition phase starts, in which words with a higher error frequency are repeated more often than well known words. The current implementation of the A-BKT model, in contrast, starts to practice some few skills first. But if the provided answers contain a lot of mistakes, the algorithm focuses on this smaller set of weak skills before it introduces new skills. This ensures that the learner has already a good basis of the first skills instead of getting overwhelmed by too much content.

After a skill is selected the system has to decide with which tutoring action it should be addressed. As mentioned before, the task difficulties are considered as tutoring actions, which represent abstract tasks that are mapped onto concrete exercises or pedagogical acts in the SARTS architecture later on. Similar to the skill selection, the action selection is modeled as a minimization problem. That is, the system chooses thenext_actionthat minimizes the difference between the predicted and the desired skill belief distribution to increase the learner’s knowledge gain for the selected skill.

next_action=argmin

al∈At [ β(S_it,al)·KLD(p(Sit+1|al),p(Sopt))] (5.9) where p(S_it+1|a_l) := ∑ om∈Ot ∑ sj∈Sit [p(sj|om,al)·p(sk|sj,om,al)],∀sk∈Sit+1 (5.10) = ∑ om∈Ot ∑ sj∈Sit [ p(om|sj,al)·p(al|sj)·p(sj) p(om,al) ·p(sk|sj,om,al) ] ,∀s_k∈S_it+1 (5.11)

Here,p(S_it+1|al)is used to predict the effect of applying the current actionalto skillSiby incorporating the KLD to compare the skill belief with the desired skill beliefp(Sopt). This procedure also substitutes the process of calculating and maximizing the expected utility for now and, hence, avoids the definition of appropriate utility values for each state. As a further simplification and as a control mechanism to

refine the process of selecting the “best” action for each state, the factorβ(S_it,al)is introduced. It is based on a series of overlapping Gaussian curves distributed over the full belief space of a skillSi, while each curve defines the field of application for a specific actional(task difficulty, see Figure5.3). Consequently,β(S_it,al)modifies the KLD so that it is higher if an action is selected, which is assumed to be inappropriate for the current skill belief (e.g., high task difficulty for low skill mastery). However, if an action is assumed to perfectly fit the current skill belief about the learner’s skill mastery,β(S_it,al) does not affect the KLD.

Overall, the presented definition of the A-BKT approach results in a model that selects an easy task if the skill mastery is believed to be low, a hard task if it is high, and medium in-between. This is the another difference to simplespaced repetitionsystems, since the difficulty of tasks does not change within those systems. However, the goal of the developed adaptation strategy is to create a feeling of flow, which can lead to better learning results (Craig et al.,2004;Hamari et al.,2016). Consequently, it strives not to overburden the learner with too difficult tasks or to bore them with too easy tasks, both of which may lead to frustration and, with that, hamper learning (Engeser and Rheinberg,2008;

Habgood and Ainsworth,2011).