The Effects of Feedback on Argumentation

The data collected through the forms of analysis described above allow to provide feedback to students and groups. Scheuer et al. (2010) identifies three important aspects of feedback in argumentation. With timing and control he refers to the circumstance when the feed- back is provided (on demand, immediately or summative) and who provides the feedback (system,peers,moderator). Second, themode and contentof feedback plays a role. In ar- gumentation systems, feedback is either displayed in atextualform, byhighlightingcertain things, especially in graph-based representations or in form ofmeters. These are mirroring or metacognitive tools (described in detail in Chapter 2). Third, the aspects of feedback selection and priorityhas to be considered, since it is essential to deliberate the amount of feedback carefully, as too much feedback might overwhelm students.

To our knowledge, the aspect of feedback has not been systematically evaluated in many studies. Scheuer et al. (2010) name two studies that investigate adaptive support, one with LARGO(Pinkwart et al., 2007) and one with a system called CONVINCEME(Schank, 1995).

Pinkwart et al. (2007) conducted a study with their system LARGO and compared it with

groups using a notepad. As stated above, LARGO is a system that integrates feedback in

form of self-explanation prompts. Their results show that for low-aptitude students, the tool provided a number of advantages over the note-taking technique. The authors suggest that one important factor for the improvement of these students is feedback. The authors could observe that the feedback functionality was increasingly used towards the end of the test, indicating that students considered the feedback as helpful. In a second study (Pinkwart et al., 2008), these results could not be replicated. The advice function was used less often and this also decreased over time. One explanation might be that students in the first study (paid volunteers) were more motivated than participants of the second study (mandatory task for students of a class without any additional benefit). However, it is not clear which of the results, if any, can be attributed to the feedback. For that, a study comparing a condition with and a condition without feedback would have been necessary.

CONVINCEME (Schank, 1995) is an argumentation support tool with which students can

create arguments and provide ratings on how strong they perceive the individual parts of an argument. Feedback is given based on a model called ECHO. ECHO is a computational implementation of the TEC model (Theory of Explanatory Coherence) (see e.g. Thagard, 1989). Students are enabled to compare their ratings with that computed by the model. In a study, Schank (1995) compared the tool with a traditional pen-and-paper method. Results show that the tool could effectively support students in structuring and also revising argu- ments. As with the study of Pinkwart et al. (2007), the authors did not isolate the factor of feedback, thus, it is not possible to estimate which part the feedback contributes to the results.

In the last two sections, related research on the two use cases for group mirrors - collabo- rative creativity and argumentation - have been discussed. In both areas, first approaches of including technologically mediated feedback have been made. These first prototypes and studies reveal the potential of such applications. In this thesis, the influence of several as- pects of technologically mediated feedback are investigated in detail. In the next chapters, a design space is presented, followed by the two main chapters presenting prototypes and studies in the context of collaborative creativity and in the context of argumentation.

II

A DESIGN SPACE FOR GROUP

MIRRORS

4

Design Space

Research on group mirror systems is a relatively novel field, and therefore lacks a shared un- derstanding of the overall design space. Jermann et al. (2001) and Streng et al. (2009) suggest two different ways of categorizing feedback systems for groups. Jermann et al. (2001) pro- pose a classification framework that distinguishes betweenmirroring systems,metacognitive toolsandguidance systems. Streng et al. (2009) define a design space consisting of thetype of information, the type of visualizationand the placement. However, these classifications only cover a certain perspective on feedback systems. The following classification provides a more exhaustive view, defining the most important characteristics of group mirrors. In the next sections, I will outline the existing classifications by Jermann et al. (2001) and Streng et al. (2009) in more detail. Besides that, I will discuss variables of the CSCW design space in regard to their influence on group mirrors. Afterwards, I will define a design space that describes the possible characteristics of group mirrors. In the last section, I will explain and motivate the design choices that have been made throughout this thesis, for example, the focus on co-located scenarios instead of distributed environments.

4.1

Previous Classifications

Jermann et al. (2001) provide a classification of CSCL systems. Their classification focuses on online tools for distributed collaboration in the context of learning scenarios. They dis- tinguish between mirroring systems, metacognitive toolsand coaching systems. Mirroring systems reflect basic actions to the collaborators or teachers. An example of a mirroring systemis a visualization of the actions students undertake in a CSCL system, for instance, a visualization of the amount of contributions in a chat. Metacognitive tools are systems that compare the current state of interaction to the desired state of interaction. This can be accomplished by visualizing the current state next to the desired state, for example, by show- ing the current number of contributions in a chat next to the desired amount of contributions. This information is then either visualized and displayed to the participants to promote self- regulation, or it is collected to be analyzed afterwards by a researcher or coaching agent.

Coaching systems use a model of interaction to interpret indicators and offer guidance. In case of the chat example this could be an advice to the collaborators on how to engage more or less in the discussion.

The classification of Streng et al. (2009) applies to group mirrors for co-located collabo- ration. While the definitions of Jermann et al. (2001) are located on a more conceptual level, the classification of Streng et al. (2009) is more concrete and application-related. The characteristics that they name are thetype of visualization, theplacementand thetype of in- formation. All these factors are possible characteristics ofmirroring systems,metacognitive tools as well ascoaching systems. However, Streng et al. (2009) mainly refer tomirroring systemsin their work.

They define thetype of visualization as either diagrammatic or metaphoric. Previous group mirror systems typically visualized information in a diagrammatic way, for example, in form of charts, such as bar charts or pie charts. An example is the SECOND MESSENGER of

DiMicco et al. (2004) that uses bar charts in its first version. Another type of diagrammatic visualizations are matrices of light sources, used in the REFLECT table by Bachour et al.

(2008). Streng et al. (2009) present the first metaphoric visualization for group mirrors. In a direct comparison of diagrammatic and metaphoric visualizations, they could show several advantages of metaphoric feedback compared to diagrammatic one. The second classification item is theplacementof the group mirror on either table or wall. Streng et al. (2009) name pros and cons for both settings. With a table, information is in the center of the group, at the price of creating an orientation problem for some of the group members. Wall displays solve the orientation problem, but might not be in everybody’s vision for groups seated around a table. The third classification part is the type of informationthat can either be quantitative or qualitative. The following design space adopts this differentiation, which will be explained in more detail in section 4.3.