Study Methodology 1 Study design

For the gestural analysis the data was generated from video recordings of collaborative interaction during several of the experiments presented and discussed in chapters 4 and 5 (specifically, section 4.2, which compared Remote Gesture vs. Voice Only Communication and section 5.3 which specifically analysed the performance effects of three forms of remote gesture production). The study design is therefore identical to that described in sections 4.2.1.1 and 5.3.3.1.

6.2.2 Participants

Participants are the same cohorts as presented in section 4.2.1.2 and 5.3.3.2.

6.2.3 Equipment

Equipment used was as reported in section 3.5 – including the various different apparatus required for the generation of differing forms of remote gesture as discussed in chapter 5.

6.2.4 Materials

Materials present during collaborative interaction were the same as detailed in the descriptions of the relevant experiments.

6.2.5 Procedure

As reported in sections 4.2.1.5 and 5.3.3.5.

6.2.6 Analysing the gestures

Having viewed the video recordings of collaborative action, common patterns of behavioural interaction and gesture use were observed and noted. The structure of these typical interactions is discussed in detail, and presented below as a series of vignettes, in each case describing the work of the gesture at that specific point and its‟ attempts to aid communication. This analysis demonstrates how bodily practices help to structure the organization of work. This form of analysis is conducted first for gestures performed during collaboration through hands only means of gesturing and subsequently with addition of a sketching facility and finally when only sketching was available (and hands were no longer visible). Given that the functions of gesturing presented significantly overlap, in terms of their prevalence amongst the differing forms of gesture production, greater emphasis is given in later stages of the analysis to those

aspects of gestural behaviour which are significantly unique to the specific gesturing medium in question.

Prior research has demonstrated that the most prevalent forms of gesture in collaborative activity are pointing gestures, but recreation of these simple pointing behaviours is not sufficient to significantly improve performance. It was therefore felt that to quantify the observed patterns of gestural use would lead to inappropriate assumptions being made about the relative contribution to task performance of the varying forms of gesture. To describe in detail the variety and possible form of these „remote gestures‟ was considered to be of more benefit. Likewise it is an oft used methodology within such research, concerning the collaborative use of gesture, to borrow classification schemes (taxonomies or typologies) from the social sciences to organize findings and inform design (see Bekker et al. 1995 for a classic example). It was felt that to make such comparisons would be largely futile. From an analysis of the video recordings it was clear that the majority of gestures utilised in an object-oriented collaboration are (as stated above) primarily deictic, and of those other gestures the majority are as McNeill would describe them, Concrete Iconics (McNeill, 1992), with a smattering of specific discourse structuring and perhaps incidental (i.e. not intentionally communicated) gestures such as Batons (Argyle, 1988) and Butterworths (McNeill, 1992). To reduce the gesture analysis to a quantification of the gestures and a resignation to use these pre- determined categories limits the interpretation of the function of the gestures. It does not help to explain what these gestures actually mean to the process of the discourse and their relative importance at various points in the communication, i.e. to know that a gesture is a concrete iconic gesture is all well and good but this does little to inform us of exactly what the gesture is trying to convey. As Adam Kendon argues:

“The various typologies of gesture that have been put forward are in part attempts to classify gestures in terms of the information they encode, albeit at very general levels. These typologies are often logically inconsistent, in many cases formed on the basis of rather hasty observation with a good admixture of „folk‟ categories thrown in ... gestures that consistently occupy extreme ends of these dimensions (with little weighting on the others) get distinguished as “types” - but I don‟t think a typological way of thinking is very helpful. Rather, it tends to obscure the complexity and subtlety [of gesture].” (Kendon 1996)

In order to develop a broader understanding of the role of remote gesture in cooperative activity the concern to characterize findings in terms of existing taxonomies was replaced with a concern to understand the „stroke of gestural phrases‟. That is, to understand what gestures „say‟ and „do‟, put simply, what the gesture is „meant for‟. Whilst some might contend that such an approach, to reject the use of pre-existing taxonomies of gesture type, will lead to the inclusion of discussion of highly idiosyncratic and therefore unrepresentative forms of gesture, it was felt that the approach of exhaustively analyzing the recorded data and analysis with

multiple observers present would enable more consistent patterns of gestural behaviour to be distilled. Despite certain idiosyncrasies between individual signalers some authors firmly believe that gestural communication can only work if there is some consistency between gesturing behaviours which enables their common interpretation. As Kendon (ibid.) puts it,

“It is often said that gesticulation is idiosyncratic, each speaker improvising his own forms. So far as I know, no one has ever really tested this claim. My own experience in gesture-watching suggests to me that people are far more consistent in what they do gesturally than this „idiosyncrasy‟ claim would lead one to imagine … [There are] similarities in the patterning of gestural action and such patterns are socially shared - hence there is conventionalization to a degree affecting all kinds of gesturing.”

The following sections present the series of vignettes that articulate the patterns of gestural phrase „at work‟ in the previous experiments and the ways in which they functioned, considering multiple forms gesture representation.9