Mutual Modeling

Individuals modeling indexes

The mutual modeling accuracy had been calculated for each individual. As described by Kenny et al. (1998), we checked the non-independence of the results through the computation of intraclass correlation. This index reflects the correlation between the evaluation of how player A estimates player B’s strategies (calculated thanks to the results of the in-game questionnaires and compared to B’s answers: MM(A,B)) and the evaluation of how player B estimates player A’s strategies (calculated thanks to the results of the in-game questionnaires and compared to A’s answers: MM(B,A)). We found a positive and significant correlation (r = .38, p < .05) between the representation B made about A and the representation A made about B.

Two conclusions can be drawn from this result. The first one is that the mutual modeling appears to be a group variable rather than a personal activity. We expected accuracy of mutual modeling to be a personal parameter, i.e. that some participants spontaneously pay more attention or engage more effort in monitoring their peer. This could be due to some social attitude or to specific cognitive skills required to build a mutual model. This strong correlation supports a different hypothesis in which mutual modeling emerges as a property of the quality of interactions among peers: some pairs

seem to collaborate in such a way that their verbal and non-verbal interactions produce more cues available to both partners so that they can build a mutual model. However, since the correlation was not equal to 1, this does not remove the fact that there is an individual variability. Besides, if we consider the presence of the MLA tool, the correlation is bigger for players who did not have the MLA (r = 0.44) than for the others who had it (r = 0.24); but the difference is not significant (F[1,13]= 0.1445, p- value = 0.7097). Perhaps this result could be explained by the fact that as the players without MLA had less information about each other, they were forced to communicate much more and to be more explicit.

The second conclusion is that this positive correlation expresses the non-independence of the results among groups. Then, when we investigate the influence of the MLA on mutual modeling, we will have to use the group as the unit of analysis.

Mutual Modeling and MLA

Hypothesis H2 predicted that the MLA would have a positive effect on the mutual modeling accuracy. Since we have to use the group as the unit of analysis, the group measure of mutual modeling was obtained by taking the mean of the mutual modeling index within a pair. This group measure was defined as MMg in Table 9. Figure 28 shows a boxplot representing this accuracy for pairs in the two experimental conditions.

Figure 28. Mutual modeling measures for the groups in the two experimental conditions (MLA: with the Mutual Location-Awareness tool, NoMLA: without).

Figure 28 shows that the means of Mutual Modeling accuracy in both conditions were very close (With MLA: m = 1.58, sd = 0.87; Control: m = 1.63, sd = 0.48), although the standard deviations are quite different. The ANOVA test invalidated our hypothesis H2: the difference between the two conditions is not statistically significant (F[1,14] = 0.02, p = .889). The representation of one’s partner strategy seemed not to be facilitated by the information conveyed by the MLA tool. But this is mainly due to

the huge heterogeneity of the control group. Our second hypothesis is then invalidated.

Mutual modeling evolution over time

The third hypothesis H3 predicted an effect of time and collaboration on mutual modeling. At the beginning of the game, the players were not familiar with each other. We postulated that playing together during two hours would enable them to improve the accuracy of their mutual modeling. Like the preceding analysis, we looked at this index at the group level (as indicated by the intraclass correlation). Figure 6 depicts the evolution of the MM accuracy for pairs in the two experimental groups.

Figure 29. Evolution of mutual modeling accuracy from level 1 to 3. The horizontal axis expresses the time discretized in 3 different levels.

As Figure 29 illustrates, the mutual modeling accuracy rose a little between the two first levels (MM1 and MM2) and then there was a increase between the two last evaluations (MM2 and MM3). As a clue to this phenomenon, we should look at the moment of the evaluation of the mutual modeling accuracy. The first measure indicated the baseline for the mutual modeling accuracy. MM2 represents it after one level of play and MM3, after two levels. The implication is that perhaps the players needed two game levels (nearly one hour) to become familiar with each other. The surge between MM2 and MM3 might be due to this phenomenon.

Repeated measures analysis showed that the effect of time on the accuracy of mutual modeling is only a trend (F = 3.189, p = .084) as depicted on Figure 29. We then cannot accept our third hypothesis: there is only a marginal effect of time on mutual modeling accuracy. Additionally, there was no effect of the presence of the MLA tool (F = 0.1, p = .75), which is consistent with the rejection of our second hypothesis. There is also no interaction between time and the presence of the MLA

tool (F = 0.05, p = .95). As a consequence, this supports the argument that the sudden increase of the accuracy of mutual modeling is not due to MLA.

Mutual modeling, MLA and collaborative task

We found a positive correlation between the accuracy of the mutual modeling and the group task performance (r = 0.42, p = 0.05). Pairs with an accurate mutual model reached better scores. Additionally, a regression analysis has shown a positive and significant relation between the group score and the mutual modeling accuracy (
= 54, p = .02). This MM index then proved to be a good predictor of the group task

performance. However, post-hoc comparisons on contrasted groups did not show any interactions between MLA, performance and mutual modeling accuracy. Unfortunately, the low number of subjects did not allow us to conduct analyses on contrasted groups to see, for instance, the impact of mutual modeling on score.

Moreover, looking at the percentage of time spent in each view (“scout” or “spaceship”) by the pairs in the two conditions as shown in Table 11, we noticed an interesting difference. Teams in the MLA condition spent more time in the “scout view” than teams in the other condition, while the average time spent in the “spaceship view” was similar in the two conditions. Since the distribution of the variable was not normal, we used a Wilcoxon test, which showed that the difference concerning the percentage of time spent in the “scout view” was significant (W = 70, p < 0.001). Since the view of the partner’s scout was the MLA tool, it was not surprising that players in the tool condition spent more time in this mode. By staying in the scout view, they could give information to their partners about their intentions and their locations.

Table 11. Average percentage of time spent in the “scout view” and in the “spaceship view” by the pairs in the two conditions.

With MLA Control

Average percentage of time spent in the “scout view”

75.41% 62.54% Average percentage of time

spent in the “spaceship view”

24.59% 37.46%

We then performed a post-hoc split of players into two kinds of participants depending on the time they spent in the scout mode. The split point was the mean of time spent in that mode and it led to the constitution of two groups made up of 12 individuals “short time in scout mode” and 24 individuals “long time in scout mode”. A two-way analysis of variance conducted on these contrasted groups revealed that pairs in the awareness condition who spent more time in the scout mode reached higher levels of mutual modeling (F = 8.02, p = 0.015) than the others. It implied that there was an effect of the MLA on the accuracy of the mutual modeling only for teams who spent a long time in the “scout mode”. Therefore, it seemed that the information conveyed by this awareness tool could be a benefit for team collaboration. This information could help players in order to estimate their partner’s strategies if the participants understood that they had to make an effort: spending an accurate time in the scout view. The teams who did not spend enough time in the scout view had no benefit of the MLA tool.

Summary of quantitative results

In sum, we found that our main factor (presence of Mutual location-awareness) had a strong positive impact on group performance of the task. It also seemed to have a positive impact on mutual modeling accuracy only for players who used it intensively. Furthermore, we discovered that the accuracy in predicting other’s intents was only marginally influenced by time; and overall that when one player in a group guessed the partner’s intents accurately, so did the second partner. Finally, the positive correlation between the accuracy of the mutual modeling and the group performance seemed to account for the benefits of the MLA. The next section aims at understanding how MLA has been used and interpreted by players in this experimental condition.