Feedback in Lectures

Technical systems used in classrooms or lectures are referred to as classroom communication systems (CCS) (Beatty, 2004). These systems can enhance communication between audi- ence and lecturer in a more unobtrusive way than face-to-face communication. An important part of these tools is the feedback that is provided to both the audience and the lecturer. A number of systems allow students and lecturers to exchange questions and answers, either as quizzes or in a more open-ended form (see e.g. Dufresne et al., 1996; Harry et al., 2009; Ratto et al., 2003; Reinhardt et al., 2012; Wessels et al., 2007; Yourstone et al., 2008), others permit different forms of note-taking (e.g. Kam et al., 2005; Simon et al., 2008; Wilkerson et al., 2005) or written feedback (e.g. Anderson et al., 2003). An overview of Audience Re- sponse Systems (ARS), also called “clickers”, has, for instance, been presented by Caldwell (2007), an overview of learning dashboards, categorized in systems supporting face-to-face lectures, supporting group work during face-to-face sessions and online or blended learning scenarios can be found in Verbert et al. (2014). As in the scope of this thesis, the interest lies more on visually provided feedback of collaborative activities, I will outline some of the systems with elaborate visual feedback in the following.

Chen (2003) presents a system for remote lectures using a multiparty videoconferencing sys- tem. Below the videos, speech activity is indicated with yellow bars, hand motion with red and body motion with green bars. Aside from that, a summary of the whole conversation is visible (see figure 2.5, top left). Results of their study reveal that teachers cherish especially replay feedback, meaning feedback that is shown following the conversation in contrast to feedback that is shown in real-time. When students were asked if they would feel comfort- able if video, audio or the activity information were recorded, most of them spoke in favor of the activity information as this maintains privacy to a higher level than video.

Sturm et al. (2006) developed a system dedicated to provide feedback about the audience to the lecturer. They used a similar color scheme, however in their system, designed for co-located lectures, these colors indicate attention level in form of a pie chart (green = ready, red = busy, grey = other) and interest level in form of a scale (red = low, yellow = medium, green = high) (see figure 2.5, top middle).

Bry et al. (2011) and Pohl et al. (2011) developed the system BACKSTAGE, a digital

backchannel used in large lectures and classes. It aims at improving awareness of both the audience and the speaker and at helping students to actively participate in the lecture.

572

Martin Ebner, Christian Haintz, Karin Pichler, Sandra Schön

geben. Der/die Vortragende sieht in seinem Webbrowser in Echtzeit das gesam- melte aktuelle Feedback seines Publikums. Dadurch soll die aktuelle Stimmung und das Verständnis des Publikums dargestellt werden, um als Vortragender unmittelbar darauf reagieren zu können.

3.3 Design

Das Design wird den Anforderungen entsprechend gewählt. Eine Anforderung ist Mehrsprachigkeit, diese wird speziell durch Einsatz von Symbolen und Bildern anstelle von Text erreicht (vgl. Abbildung 2 mit dem Display für das Publikum).

Abb. 2: Screenshot der Teilnehmeransicht des Backchannel-Prototypen

Auf der linken Seite in Abb. 2 sieht man die Eingabeansicht für die Teilnehme- r/in. Hier kann mithilfe von drei Schie bereglern die aktuelle Be ndlichkeit über- mittelt werden. Die rechte Seite stellt das Gesamtbild aller Teilnehmer dar. Alle Änderungen werden in Echtzeit dargestellt. Die Rückmeldung der Teilnehmer erstreckt sich dabei auf drei gewählte Feedback-Dimensionen (Zufriedenheit, Verständnis, Vortragsgeschwindigkeit) die alle durch Symbole erklärt werden sollen. Die Feedbackwerte der drei Dimensionen ergeben die Mimik und Gestik des Avatars, der den jeweiligen Zustand darstellt (siehe Abb. 3).

Development of Peripheral Feedback to Support Lectures 147

OTHER 32% BUSY 16% READY 52%

Fig. 2. Visualization of attention level (left) and interest level (right) percentage of students that have the application as the active window, but are not actually using it in green (“READY”). Green in this case means that the teacher may move on, whereas red denotes students who are not yet ready to move on. The idea is that, if the BUSY area is small relative to the READY area, the teacher may move on, but if the BUSY area is large relative to the READY area, s/he might wait a moment before moving on until more students finished their notes. The grey area (or “OTHER”) indicates students who are doing other things on their notebooks, such as chatting or browsing the web. This category is assumed to be irrelevant to the teacher’s decision to move on or wait. The current Interest level is represented as a scale, with red indi- cating a low interest level and green indicating a high interest level and is calculated across BUSY and READY students (as said before, students doing other things are con- sidered by definition not to be interested in the lecture). The horizontal line indicates the average interest level since the start of the lecture. Both visualizations are updated dynamically in real-time. The update rate of the information will be optimized by experimentation, so as to make sure that the information accurately and meaningfully reflects the current situation, but is not too distracting for the teacher.

3.3 Technology

As mentioned before, for measuring attention level we distinguish three groups of students: students who are actively taking notes using the Agilix GoBinderTM _applica-

tion, students who have the note-taking application as the active window, but are not actively using it and students who are using other applications on their laptop. To collect this information, monitoring software will be installed on the students’ note- books, which provides information about the application which is currently active or has been used most recently. The individual information is sent to a central server in real-time; the server collects the information and changes it into a graphical represen- tation that is sent to the teacher’s laptop.

Interest level can be measured in many different ways. Initially, we will ask the students to actively provide information about their own level of interest by means of a slider bar that they can adjust using their laptops (such as in [10]). In the future, we

Figure 2.5: Related work on feedback in lectures. Top row from left to right: video confer- encing system by Chen (2003), feedback about the audience by a system of Sturm et al. (2006), a feedback system for the classroom by Bakker et al. (2012). Lower row: ACTIVITYAGGREGA- TORby Pohl et al. (2012) in the lean version (left) and extended version (middle) and a feedback system by Ebner et al. (2014).

One of the main functionalities is a microblog that enables students to exchange comments and ideas. This can be done publicly or privately as well as anonymously or by using pseudonyms. Students have the possibility to categorize and rate their messages. The best rated messages will then be displayed to the lecturer. The lecturer in turn has the possibility to conduct short quizzes. In a first study, Pohl et al. (2012) could show that with BACK- STAGE, more questions were raised during lectures than without the system. Furthermore,

the user study revealed that the system could be improved by integrating presentation slides. This was included in an enhanced version (see e.g. Baumgart et al., 2011; Gehlen–Baum et al., 2012).

Pohl et al. (2012) present a further iteration of the interface. Additionally to the microblog and the slides, there is a space for the virtual representation of the lecturer and awareness components. The interface is designed to support both social as well as workspace awareness (for more details on awareness see Section 2.3). Social awareness is promoted through an indication of the presence of the user (online, offline or busy). The display of certain aware- ness information is dependent on the mode. Students in busy-mode who want to concentrate on the lecture are only provided with the slides and quizzes. The ACTIVITY AGGREGATOR

is a visualization that displays the activity level derived from writing and rating of messages for the student and the audience. The lean version is designed for using it during the lecture (see figure 2.5, bottom left), the extended version as a replay (see figure 2.5, bottom mid- dle). Concepts to support workspace awareness are, for example, the categorization of posts,

2 Feedback

which reveals strongly debated topics, a notification mechanism that shows all activities related to a student’s post, or rating and ranking of the posts. The annotating functional- ity provides awareness information through the position of the annotation on the slide and through the chosen category. In another study, Gehlen–Baum et al. (2014) compared the use of mobile phones in lectures with and without the support of BACKSTAGE and could show

that the system helped students to stay involved with lecture-related activities.

An example of a tool envisaged for the use in classrooms to support communication between teacher and students is the system FIREFLIES by Bakker et al. (2012). It is composed of

light objects placed on the students’ desks and a teacher-tool with which the teacher controls the light objects by switching between four different colors (see 2.5, top right). Auditory feedback can additionally be provided about the color of the objects (e.g., ocean-sounds representing yellow, owl-sounds representing green). Bakker et al. (2013) evaluated their open-ended interactive system in a study in four different classrooms and observed that it was used in different ways to provide feedback. It was, for example, used to give compliments or to give a turn to a student. In general, the visual feedback was perceived as more successful than the auditory feedback.

Ebner et al. (2014) present a system with which students can provide feedback about satis- faction, understanding and pace autonomously to the lecturer using an interface with sliders. The value of the aggregated feedback is represented through facial expressions and gestures of avatars of the student and of the audience (see figure 2.5, bottom right).

In summary, visual feedback that is provided in lectures to either students or lecturers of- ten uses color coding to transmit information (see, for instance, also a tool presented by Maldonado et al., 2012). Other mechanisms for coding feedback information are using tra- ditional visualization methods such as scales, bar charts or pie charts. One system uses scales in combination with avatars that represent the students. In contrast to group mirrors, these systems are designed for the specific context of the classroom or lectures, in which a clear differentiation between the teacher or lecturer and the students is made. In most cases, different information is presented to the lecturer and to the students.