Evaluation Results

After having done the surveys, we started to evaluate the data. As mentioned in the sections before, we applied a t-test to the data. Its independent variable is the type of the video, having two possibilities: standard lecture recording with one fixed video source and enhanced lecture recording with four video sources. The dependent vari- ables are: attentiveness, interest in the video, three aspects of the motivation scale based on (Rheinberg, Vollmeyer & Burns, 2001), namely QCM interest, QCM prob-

As mentioned above, we had 33 participants in our evaluation study, distributed over the two types of lecture recording. The standard lecture recording was watched by 17 attendees while the enhanced lecture recording was watched by 16.

The following tables show the results of the t-tests for every one of the six aspects we evaluated. In addition, we present the average and the standard deviation as charts separated by their underlying scales. While the aspects attentiveness and interest in

the video are based on a numeric scale from 1 to 4, all QCM aspects are based on a

numeric scale from 1 to 7, and the learning gain is based on a numeric scale from 0 to 20. Therefore, we present them in three different figures.

Table 15: t-test results for the construct "attentiveness".

Condition Average Std. Dev. t-test P

enhanced recording

2.24 0.713

standard recording

2.46 0.670

-0.920 _{not significant}

• No significant differences between both types of lecture recording.

Table 16: t-test results for the construct "interest in the video".

Condition Average Std. Dev. t-test P

enhanced recording

2.09 0.957

standard recording

2.01 0.831

0.254 _{not significant}

• No significant differences between both types of lecture recording.

When looking at Figure 39 both aspects reflecting the interest and the fascination in the videos are rated almost equally, and both have a high standard deviation. While the average of the attentiveness of the enhanced video is a little lower than the one of the standard video, the average value of the enhanced lecture recording concerning the interest in the video is slightly higher than the value of the standard lecture recording video.

1 2 3 4

Attentiveness Interest in the video

Enhanced recording Standard recording

Figure 39: Average values of attentiveness and interest in the video compared. QCM interest

Table 17: t-test results for the construct "QCM interest".

Condition Average Std. Dev. t-test P

enhanced recording

2.95 1.152

standard recording

3.25 1.476

-0.641 _{not significant}

• No significant differences between both types of lecture recording.

QCM probability of success

Table 18: t-test results for the construct "QCM probability of success".

Condition Average Std. Dev. t-test P

enhanced recording

3.95 1.69

standard recording

4.19 1.45

-0.435 _{not significant}

QCM anxiety

Table 19: t-test results for the construct "QCM anxiety".

Condition Average Std. Dev. t-test P

enhanced recording

3.31 1.501

standard recording

3.10 1.541

0.359 _{not significant}

• No significant differences between both types of lecture recording.

The results in Figure 40 show almost the same information as the figure before. Con- cerning the aspects of QCM interest and QCM probability of success, the enhanced lecture recording video is rated to be on the average slightly below the average value of the standard lecture recording video. Consequently, concerning the aspect of QCM anxiety, the enhanced video average value is slightly higher than the one for the stan- dard video. 1 2 3 4 5 6 7

QCM interest QCM probability of success QCM anxiety

Enhanced recording Standard recording

Learning gain

Table 20: t-test results for the construct "learning gain".

Condition Average Std. Dev. t-test P

enhanced recording

9.81 3.48

standard recording

9.12 3.36

0.584 _{not significant}

• No significant differences between both types of lecture recording.

Again, there is not much difference in the average learning gain between both types of lecture recording. The figure shows a slightly higher average learning gain when watching the enhanced recording of our Automatic Lecture Recording system.

0 2 4 6 8 10 12 14 16 18 20 Learning gain Enhanced recording Standard recording

Figure 41: Average values of learning gain compared.

We amend this result by some more aspects concerning the motivation and the self- assessment of the attendees. At first, let us take a closer look at the motivation of the attendees. Are they unconcerned or are they sufficiently motivated? In our question- naire, we asked five questions having a scale from 1 (does not apply at all) to 4 (fully

Table 21: Statistic results of the items concerning the motivation.

I watched this video because € Average Standard Deviation

• I have been payed for attending. 3.48 0.906

• I wanted to support the research in this area

by attending. 3.12 0.893

• its content is important for my future career. 1.18 0.465

• I am interested in the technology presenting

the learning matter. 1.76 0.902

• I like learning multimedia-based. 2.15 1.034

The type of lecture recording is insignificant for the motivation of the attendees. Therefore, we evaluated the items at once for all 33 participants. They all were extrin- sically and intrinsically motivated, as the first two items show. In contrast, neither the technology nor the content shown was relevant to the participants. Even the relatively new technique of „multimedia-based learning… is only a moderate reason for attending the evaluation study.

Furthermore, it is of interest how the attendees rate their own ability of learning using such a medium. We evaluated the difference between the two types of lecture re- cordings by two items, the first having a scale from 1 (one time), 2 (1 to 2 times), 3 (4

to 5 times) to 4 (more than 5 times) while the second has a scale from 1 (nothing), 2

Table 22: Assumed number of repetitions of lecture recordings.

Average Standard

Deviation

How many times do you think you have to watch this video in order to perceive the entire learning matter of the lecture?

Enhanced: 2.44

Standard: 2.29

Enhanced: 0.727

Standard: 0.686

It is interesting that the attendees thought they would need slightly more repetitions of the lecture recordings of the enhanced video than of the standard video although they already had a slightly higher learning gain after having watched the video once. The self-assessment of the attendees before the post-test is shown in Table 23.

Table 23: Assumed learning gain.

Average Standard

Deviation

According to your opinion, how much did you learn from this lecture by this kind of video direction?

Enhanced: 2.63

Standard: 2.53

Enhanced: 0.719

Standard: 0.514