Recall of partners’ trails

As in the previous experience, we measured the number of errors between the path produced by player A when drawing B or C to B or C’s real paths. This individual index represents the quality of A’s representation of B and C’s behavior in space. We checked the non-independence of the results through the computation of intraclass correlation (r = 0.70), which was not significant (p < .001). This expresses the non- independence of the results among groups. It means that the number of errors made by the subjects is dependent on the number of errors made by the partners (i.e. if one

player made a lot of errors about his/her path, the same goes for the partners). It also led us to use the group as the unit of analysis. To do this, we added the number of errors made by each individual within a group to create a group index. Figure 40 shows the number of errors in each condition per group.

Figure 52. Number of errors made by each participant during the post-test (while drawing the path of the partner) in the two experimental conditions.

Players in the Control condition made more than half the errors of those who had the Plan (Control: m = 33.90, sd = 17.3; Plan: m = 14.56, sd = 6.58). A non-parametric test showed that this difference was significant (W = 655.5, p < .0001). In other words, not giving a plan to the players significantly diminished the accuracy of the player’s representation of their partners’ trails. This is what we expected in our second hypothesis.

6.3.3

Communication through shared map annotations

Annotations frequency

Map annotations have been investigated both by quantitative measures like the frequency and content categorization. This variable had been studied at the individual level since the intraclass correlation among the group was not significant (r = 0.29, p = .09). Figure 53 shows the frequency of messages sent by each player in the two experimental conditions.

Figure 53. Frequency of shared map annotations written on the Tablet PC by each individual.

The frequency of map annotations was higher in the Control condition (m = 7.47, sd = 4.94) than in the Plan condition (m = 4.07, sd = 2.83). A one-way ANOVA test showed that this difference was significant (F (1, 54) = 8.2096 p < .05): players who had not had discuss a plan face-to-face wrote fewer shared map annotations.. Our third hypothesis is then validated.

We coded the content of the messages and their pragmatic status using the same coding scheme described in the previous chapter. Statistical analyses12 _{showed that the}

players in the Control condition sent more messages about strategy (W = 690.5, p < 0.0001) orders (W = 524, p-value = 0.03490) and more acknowledgements (W = 540, p-value = 0.009117). We did not find any significant correlation between these variables and performance or position recall indexes.

Having the MLA interface, all groups (with or without plan) did not discuss their own position and direction because the tool conveyed this information. However, it seems that players from the Control condition who had no opportunity to discuss strategy issues took better advantage of the annotation capabilities to share messages about strategy. The next part will further our understanding of how coordination occurred in

12 _{Given that the distribution of the data was not normal, we used a non-parametric test (Wilocoxon’s}

the different conditions by looking at the exchange of communication primitives over time.

Coordination devices use over time

Based on the seven communication primitives we listed in Section 6.2.2, we can count when the players used them, for each group in the two experimental conditions,. Of course, sometimes these primitives were not sufficient to convey meaning and players employed more verbal comments. We also counted these verbal descriptions or questions that addressed strategic issues and division of labor. We have represented in Figure 54 the exchange of these communication primitives in each of the three phases of the game. Based on the qualitative analysis of the previous experiment, the figure also shows which inferences can be performed based on these communication primitives. The point of this figure is to depict the total number of messages exchanged at a particular moment by each group, and to show the potential differences between conditions. We counted the symbols accurately reproduced and those different from the ones proposed in the palette. Overall, the percentage of symbols accurately reproduced by players from the Control condition is 52% and 73% for those from de Plan condition. This difference might be explained by the fact that players from the Control condition needed to discuss the strategy more thoroughly and symbols are too limited for this.

Figure 54. Exchange of coordination devices over time, with the main differences between experimental conditions expressed as boxes (numbers under the boxes indicates the Kruskall Wallis W as well as the p-value to show the significance of the test). The numbers for each coordination device is the sum for all groups. We also indicated the inferences each coordination devices allowed players to perform, based on the results from the last experiment.

The most striking result is certainly the unbalanced repartition of messages over time. As we saw in the preceding section, the groups in the Control condition sent more messages. The time analysis here shows that the difference was significant only for the first phase of the game (W = 564.5, p < .01): players from the Control groups sent more messages in phase 1 (m = 3.23) than those from the Plan condition (m =

0.76). And we found no significant differences for phase 2 (W = 506.5, p = 0.09) or phase 3 (W = 424 p = 0.73). This result matches our expectations: since the players did not have the opportunity to discuss the plan, they needed the first phase of the game to compensate for this by using the shared map annotations.

As one can see on this figure, in phase 1, players in the Control condition exchanged more messages about position (m = 0.9, W = 69.5, p < 0.05), the location of the object (m = 0.43, W = 67, p < .05) and strategy information (m = 0.83, W = 654.5, p < 0.01). In phase 2, there were only significant differences for messages about signal strength (m = 0.23, W = 63, p < .05) and strategy information (m = 0.6, W = 75, p < 0.005). Interestingly, when we look at the repartition of coordination, what is also striking is that the differences between the two experimental conditions tend to diminish over the course of action. This might be due to the last subtask, which is different and less bound to coordination devices exchange, as we saw in the previous experiment.

In the first phase, players in the Control condition exchanged more information about the proximity sensors than those in the Plan condition. They indeed dropped more messages about the figures they read on the proximity sensors and they left these messages at their position at the time. These messages also allowed partners to infer their positions (which were already conveyed by the MLA tool anyway). Without a plan, in phase 1, there were also more messages about the objects’ location (drawn as a circle); this can be explained by the fact that they had more difficulties in locating the objects, since they were still discussing how to solve the problem through strategy messages.

As shown on Figure 54, most of the differences between the exchanged coordination devices occur in the first phase, some remain in the second one and the third phase shows very similar patterns of communication. This leads to two possible explanations: On the one hand, even though the three phases were different in terms of task content, the absence of the plan seemed to be compensated with more messages after two game phases. On the other hand, the last phase, as we saw in the last chapter, did not require a lot of verbal communication, which can explain why the two conditions showed similar patterns of coordination device use.

Strategy messages

The previous experience has shown the importance of strategy messages for mutual modeling and task performance. The analysis described in the previous section also shows an interesting difference between players who had a plan and those who did not concerning the exchange of strategy messages that were not conveyed by the communication primitives, especially in phase 1 and 2. We looked at their repartition in both of these phases. Figure 55 depicts the number of strategy messages sent by players in both conditions.

Figure 55. Boxplots showing the number of strategy messages verbally described (not using the communication primitives) by players in the phase 1 and 2.

Groups who had a planning phase sent almost no strategy messages, which is consistent with the second study. For Control groups, the absence of a plan was compensated for by the exchange of strategy messages in the first and second phase but things are more complex. What these boxplots highlight is that the variations of behavior within Control groups are important: some of them exchanged many coordination devices and some did not.

Furthermore, there is a strong and significant correlation between the number of strategy messages in the first phase and the task performance (r = 0.50, p = 0.02), which is not the case for the second phase. This correlation might explain the broad range in performance for groups in both conditions: better teams exchanged more coordination devices in the form of strategy messages. However, since the groups in the Plan condition shared almost no strategy messages, this correlation is more pertinent in explaining the broad variety of performance for the players in the Control condition.

Finally, in terms of the pragmatics of the strategy messages sent by players in the Control condition, we found that they were mostly based on announcements and questions acknowledged by the partners. Players, for instance, proposed various strategies like staying on one level or spreading rapidly around a building. However, they never tried to state conventions; the coordination devices they exchanged through these strategy messages were, in Clark’s words, only explicit agreements.