Dependent variables

1) Learning Gain: The learning gain was calculated as the difference between the indi-

vidual pretest and posttest scores. The minimum for each test was 0 and 10, and the maximum for the pretest was 9 and for the posttest was 10.

2) With-me-ness: We used the same method as described in Chapter 6, to calculate stu-

dents’ with-me-ness levels, in this experiment, in real time.

6.5 Results

Feedback and Learning Gain:We observed a significant improvement in learning gain for

the experimental group over that for the baseline group (t (df = 49.88) = -2.50, p = .02, figure

6.2a).

Table 6.1 – Mean and standard deviations for learning gains across conditions.

Condition Number of participants Mean Std. dev. Baseline 50 0.38 0.15 Experimental 27 0.47 0.16

Immediate effect of feedback on Gaze: We observed a significant improvement in with-me-

ness levels for participants (within the experimental group) before (mean = 0.31, sd = 0.08) and after (mean = 0.57, sd = 0.16) displaying the feedback (F [1, 26] = 310, p < .001, figure 6.2b). The time difference between the moments before and after displaying the feedback was usually 2 seconds.

Overall effect of feedback on Gaze:In order to find the overall effect of the feedback on the participants’ gaze, we divided the whole video in one minute episodes. Results from a linear mixed effect model showed that on average, participants’ with-me-ness increased by 1% every minute. This improvement was significant over time (F [1, 26] = 32.60, p < .0001). Table 6.2 shows the summary of linear mixed effect model with time and participant ID as fixed and random effects respectively. Figure 6.2c shows the temporal evolution for the difference between the mean observed with-me-ness and the baseline with-me-ness for the participants; and the average number of time the feedback was shown to the participants. We can see in figure 6.2c that, towards the end of the video, the difference increased and the number of feedback displays decreased. This showed that the participants became more aware of the fact that they should follow the teacher in an efficient manner in order to learn.

6.6 Discussion

There was a significant improvement in the learning gains for the students in the experimental condition than the baseline condition. We could conclude that the gaze aware feedback helped 88

6.6. Discussion 0.35 0.40 0.45 0.50 Experimental conditions Lear ning gain (nor malised betw een 0 and 1) Baseline group Experimental group n=50 n=27

(a) Learning gain for the experimental and baseline conditions. 0.0 0.2 0.4 0.6 0.8 1.0 Feedback timing With−me−ness le v els 1.Before feedBack 2.After feedBack n=27 n=27

(b) Immediate effect of feedback on with-me-ness.

0.0 0.2 0.4 0.6 0 5 10 15 Time (minutes)

(c) Overall effect of feedback on the gaze. The whole video was divided into one minute episodes. The red curve shows the difference between the observed and baseline with-me-ness (smoothened using a two minute rolling window). The bars denote the number of feedbacks per partici- pant per minute.

Chapter 6. Gaze Aware Feedback: Effect on Gaze and Learning

Table 6.2 – Linear mixed effect model with time and participant ID as fixed and random effects

respectively.

Mean Std.

error t-value p-value Intercept 0.19 0.03 6.94 <.01

Time 0.01 0.002 5.71 <.01

the students to learn more. However, this result has to be treated carefully, as the populations were largely similar (the participant recruitment was done using the same university channel, and there was no drastic changes in student populations) in the two conditions, however the two groups of students were in two different years of the university education (the two studies were conducted one year apart from each other).

We found a significant immediate effect of the feedback on participants’ gaze. The with-me- ness levels were significantly higher after showing the feedback than those before showing the feedback. One plausible explanation emerged from the salient nature of the feedback. Since the red rectangles appeared as a salient visual feature for the participants, their attention was drawn towards the feedback.

However, the significant long term effect on the with-me-ness indicates that the feedback had an effect on participants’ attention in the terms of “how well they follow the teacher in both the deictic and dialogue spaces”. One plausible interpretation of increase in with-me-ness over time, could be, that the participants became more aware of the fact that following the teacher during is important to understand the content and they started following the teacher more closely than before. This effect is also evident from the figure 6.2c. We can see that the difference between the baseline with-me-ness and the observed with-me-ness was higher during the second half of the video.

Concisely, we could say that the gaze aware intervention in the learning process of the students was observed to have a positive effect on their attention. Provided that such a feedback is used during regular MOOC studies, this might have a long term impact on students’ overall attention. In terms of our general research question about “how to improve the attention of the students during MOOC videos”; gaze aware feedback emerged as one of the positively influencing intervention.

Our way of providing gaze-aware feedback to students has a key limitation in terms of pre- processing required. The computation of with-me-ness requires us to know all the deictic gestures and to transcribe the dialogues beforehand. This might be overwhelming for longer videos. One way to overcome this issue is to use the heat-maps to convey the content coverage and provide feedback to the students about their gaze patterns.

7

Effect of Displaying the Teacher’s

Gaze on Video Navigation Patterns

7.1 Introduction

In previous chapters, we have shown the importance of following the teacher in achieving high learning outcomes. The gaze-measure “with-me-ness” was found to be correlated with students’ learning outcome. We used the gaze as a measure of attention and a way to provide feedback to the students. The gaze-aware feedback was shown to be effective in terms of both the gaze patterns and the learning gain of students. In this chapter, we addressed a different question; “can we use gaze as a tool to drive attention?” One way to improve students’ learning experience could be to make teachers’ discourse easy to follow by augmenting additional information on the video lecture. In this experiment, we chose to augment the video lecture with teacher’s gaze and use students’ navigation patterns to quantify the ease of following teacher’s discourse.

To address the question, whether we could use the teachers’ gaze to help making the learning process efficient for the students, we augmented the teacher’s gaze on a MOOC video on Coursera (this was not an experiment in the lab). We then collected the MOOC logs containing the video navigation patterns; and analysed the data to find the effects of displaying the teacher’s gaze on the video navigation patterns of the students.

In this chapter, we show that displaying teacher’s gaze in a MOOC video-lecture could help the students understand more easily the content of a MOOC video. Moreover, this effect remains consistent with the increasing complexity of the situation explained by the teacher.

7.2 Context