Mixed-Fidelity Prototype Study (Side-by-Side vs Face-to-Face)

As the display switch did not seem to be the crucial factor for the increased amount of argu- ment sequencing in the ArgueWall condition, other factors must have played a significant role. As mentioned before, the first mixed-fidelity prototype study indicated that the awareness of the learning partner’s arguments has an effect on argument sequencing. In the second mixed-fidelity prototype study where the participants sat face-to-face this effect was not observed. Therefore, the amount of argument sequences in the one display conditions of both studies were quite differ- ent: In the first mixed-fidelity prototype study (side-by-side) the numbers of argument sequences in the side-by-side ‘one display’ condition was M=4.20 (SD=2.30, cf. section 4.12) compared to M=3.20 (SD=1.75, cf. section 4.15) in the second mixed-fidelity prototype study (face-to-face, see Figure 4.17). This supports the hypothesis that the seating arrangement and in particular the readability of the partner’s arguments plays a major role. To verify this hypothesis the effect of the seating arrangement on argument sequencing was examined in one last study8.

Eight dyads (N=16) participated in the study. 44% of the participants were female. Their age ranged from 20 to 26 (23 on average). The learning task and procedure was the same as in the study before (see section 4.3.1). However, this time the seating arrangement was varied. In one condition the participants sat face-to-face, in the other condition they sat side-by-side. In both conditions the individual phase was done on the tablet paperboards. Thus, the conditions are equal to the display switch conditions of the previous two studies respectively.

84 4 Choosing a Display (Environment)

Mixed prototype follow-up study 1 Mixed prototype follow-up study 2

4.20 3.20 0 1 2 3 4 5 6

One display Display switch

# l ink s betw een arguments _3.20 3.00 0 1 2 3 4 5 6

One display Display Switch Figure 4.17: Comparison between mixed prototype follow-up studies 1 and 2.

Results show that the number of argument sequences was higher in the side-by-side condition (M=3.50, SD=1.69) compared to face-to-face (M=2.38, SD=1.41, see Figure 4.18). A paired- sample t-test showed a significant effect (t(7)=2.55, p=.04, two-tailed, r=.69). Subjective assess- ment through questionnaires confirmed that thereadabilitywas better when sitting side-by-side (M=4.94, SD=0.25) compared to face-to-face (M=2.38, SD=1.15). A Wilcoxon signed-rank test showed a significant effect (Z=-3.45, p=.001). In addition, the question whether thespatial ar- rangement of arguments facilitates argument sequences was significantly more agreed to in the side-by-side condition (M=4.25, SD=.86) than in the face-to-face condition (M=3.63, SD=1.36, Z=-2.41, p=.02).

Mixed prototype follow-up study 3 (Susanne Zirker)

3,50 2,38 0 1 2 3 4 5 6 Side-by-side Face-to-face # lin ks be tw ee n arg ume nt s 1 2 3 4 5 readability spatial arrangement Side-by-side Face-to-face Li ke rt sca le fro m 5 (a gre e) to 1 (d isa gre e)

Figure 4.18: Number of links between arguments (left) and subjective results (right) in the Mixed-Fidelity Prototype Study 3.

4.3.5

Summary and Discussion

This section examined different display settings for an application that supports co-located col- laborative learning. It implements two collaboration scripts that effectively foster argumentative knowledge construction. Based on findings from previous studies in the field of human-computer interaction, our first vision was a tabletop application (‘ArgueTable’). However, an iterative de- sign process with small case studies led to a display environment consisting of laptops and a shared interactive wall display (‘ArgueWall’). A study showed that the ArgueWall mitigated

4.3 Follow-Up Studies 85

a problem that occurred on the ArgueTable through all case studies: the lack of argument se- quencing. We hypothesized the display switch between individual preparation on laptops and collaborative phase on the wall display to be the cause of the improvement as the display switch was the motivation for the ArgueWall in the first place. However, ArgueTable and ArgueWall differ in various factors such as the text input method, display size, resolution and orientation. We therefore conducted several follow-up studies to narrow down the cause of the effect.

The studies produced findings on different levels, which are addressed in the following three subsections. At first the difficulties of producing generalizable results are illustrated. Second, the usefulness of iterative design processes is discussed. The focus of both discussions is on the number of argument sequences, a question which we got tied up over and which caused the series of follow-up studies. In the end, the findings that were revealed despite the study design dilemma are summarized.

The Study Design Dilemma

The series of studies unfolds several problems that go along with a comparison of display envi- ronments. First, it is extremely difficult to isolate single display factors. For instance, different displays have different text input capabilities (e.g. hardware keyboards on laptops with haptic feedback vs. on-screen keyboard or handwriting recognition on interactive surfaces without hap- tic feedback). As a consequence the time and effort spent on text input can vary across conditions, leaving more or less time for the actual learning task (in our case argument sequencing). This can in return lead to further consequences, which may be difficult to interpret. The problem of comparability also applies to fairly similar display types. Even the comparison of vertical and horizontal interactive surfaces is not straightforward although they provide the same input and output technologies. Despite the technological equivalence there are significant differences such as ergonomic aspects or the problem of reach, which naturally limits the size of tabletop displays. The size of the display can have various consequences, for example on the overview or the num- ber of items that are created. In our case this would be the number of arguments, which again can affect the number of argument sequences. This throws researchers into a dilemma: For instance, in a clean study design the size of the vertical display needs to be deliberately downsized to match the horizontal display in order to isolate the factor ‘display orientation’. However, in this case the capabilities of the vertical ‘lab’ display do not reflect the capabilities of larger, more realistic wall displays as they would be used in schools.

Due to this dilemma we chose a two-fold approach. We started with a less clean study design by comparing two display environments (ArgueTable and ArgueWall). Neither of them was artificially adapted for the study. As a consequence they varied in several factors. Afterwards, we tried to narrow down the cause of the effect with artificial prototypes in several follow-up studies. However, although each of the follow-up studies specifically investigated one isolated factor, it still turned out to be difficult to produce generalizable effects. For instance, while the handwriting solution used in the follow-up studies was deliberately chosen to avoid a bias caused by different text input methods, it might have amplified the text orientation problem. As handwriting is generally harder to read than typed text, it is not sure whether the effect also

86 4 Choosing a Display (Environment)

applies to the functional ArgueTable prototype, which recognizes the handwriting and displays block letters. This series of studies depicts a complex of problems, which put into question whether effects of display types or environments on collaborative processes can be generalized.

Iterative Design Process

It is hardly possible to produce general recommendations regarding the choice of display envi- ronment. The only alternative for determining the ideal display environment for one’s learning application is going through an iterative design process. While the iterative design process of the ArgueTable led to an improved display environment, the ArgueTable studies also raise the question whether small-N case studies can reveal all effects on collaboration. For instance, the role of the seating arrangement was not identified during the iterative design process although the seating arrangement was specifically investigated in one of the case studies: Even though the orientation problem in the face-to-face seating arrangement was acknowledged, it was not con- sidered to be disturbing because the arguments were read out loud. In the end the consideration of all benefits and drawbacks of both seating arrangements spoke in favor of face-to-face. There are two possible explanations. One is that the significance of the problem was not discovered due to the small sample size. Another explanation is that the consequences of the text orientation problem were less severe on the functional prototype case study because of the printed letters. In this case we still do not know what the main cause of the effect on the number of arguments sequences in the ArgueTable vs. ArgueWall study was. It may have been a combination of dif- ferent factors. It could, after all, still have been the display switch but the effect did not occur with the mixed-fidelity prototype due to a lack of affordance (as with the paper prototype). To sum up, although iterative design processes may not reveal all effects, we still argue that (a) it is worthwhile because it led to an improved display environment and (b) it is a more suitable approach than attempting to produce general display recommendations.

Findings

Despite the design dilemma the studies revealed several findings. First and foremost, the studies showed that the display environment can play a significant role in computer-supported collabo- rative learning. Even though we were not able to identify individual display environment factors as cause of the effect on argument sequencing, there are two aspects, which seemed to be impor- tant. Both are problems related to text on tabletop displays, which turned out to be severer than expected. The first one is the text orientation problem, which exists in face-to-face seating ar- rangements. Although this issue has been addressed before (for instance by Kruger et al., 2003), we underestimated its impact because participants neither placed great importance on it in case studies nor did we observe significant effects in video analyses. Second, text input on tabletop displays using both an on-screen keyboard and handwriting recognition were so unsatisfactory that negative impact on the learning process cannot be ruled out. Again, the text input problem has been addressed before (for instance by Hinrichs et al., 2007) but we underestimated its signif- icance. Considering these issues it seems questionable whether tabletops are ideal for text-based collaborative learning tasks.