Reading Guide to Experimental Chapters

Table3.3is designed as a reading guide to the upcoming experimental chapters. The three telecom- munication experiments are presented, together with two oculesic model experiments. The chapter in which each may be found is shown in the rightmost column. The System column refers to the VR sys- tem (CAVE or WALL) used in the experiment, and the presence of a VMC comparison condition. The number listed in this column indicates the number of interactants in an experimental session. For in- stance, in the truth and deception experiment, two-party AMC between a user located in the CAVE, and a user in the WALL is investigated, together with a VMC comparison condition. The column headed EyeCVEindicates the degree of maturity of the system in terms of performance, eye tracking ability, and avatar subsystem in accordance to Section3.1. Performance of Basic maturity is comparable to the evaluation documented in this chapter, while the system’s core performance is significantly improved in

3.3. Chapter Summary 111 Matureand Mature++ levels. In particular, Mature++ transmits avatar update messages at significantly higher frequency and sensitivity. This maturity is also reflected in the following two columns entitled Eye Trackerand Avatar, which refer to the eye tracking device and avatar type employed during the ex- periment respectively. Implications of these components of the EyeCVE system have been documented in this chapter. Finally, the Analysis column refers to the methods of data analysis performed in each experiment. A variety of approaches are used, including pre- and post-experimental questionnaires, eye tracking data alone (i.e. not multimodal), task performance, conversation analysis, subjective ratings of visual stimuli, and multimodal analysis as documented in this chapter.

In summary, the three telecommunication experiments are reported in chronological order, with increasing capability with regards to both performance of EyeCVE as a telecommunications medium, and to data analysis methods. An initial experiment on three-party conversation is presented in Chapter 4, followed by a three-party object-focused experiment in Chapter5, and finally a highly interpersonal scenario studying truth and deception in Chapter6. The behavioural modelling work and associated ex- periments are presented in Chapter7, and represent work that was carried out, chronologically, between the object-focused and truth and deception experiments.

Table 3.3: Overview of experimental chapters. Information includes VR system in use, maturity of EyeCVE, eye tracker in use, avatar type in use, and analysis methods.

FOCUS& CHAPTER SYSTEM EYECVE EYETRACKER AVATAR ANALYSIS

EyeCVE Evaluation CAVE Basic MobileEye Basic - System

Section3.1 CAVE performance

Conversation CAVE / VMC Basic MobileEye Basic - CA

Chapter4 CAVE / VMC - Eye tracking

CAVE / VMC

Object-Focused CAVE Mature ViewPoint Basic - Performance

Chapter5 CAVE - Questionnaires

CAVE - Multimodal

Truth & Deception WALL / VMC Mature++ ViewPoint Advanced - Multimodal

Chapter6 CAVE / VMC - Questionnaires

- Subjective rating Oculesic Behaviours HD display N/A N/A Advanced - Performance

Section7.1 - Subjective rating

Eyelid Model CAVE Mature++ ViewPoint Advanced - Subjective rating Section7.2

Gaze Model CAVE Mature++ ViewPoint Advanced - Subjective rating Section7.3

112

Chapter 4

Experiment: Three-Party Conversation

The overarching goal of the telecommunication experiments documented over the following three chap- ters is to investigate the use of eye tracking in AMC for both interactive and analytical applications. Each chapter addresses a particular scenario that represents a common usage of, or issue central to, remote work and collaboration. The current chapter investigates multiparty conversation, Chapter5examines object-focused interaction, and Chapter6explores truth and deception.

The experiment presented in this chapter investigates three-party conversation in tracked gaze AMC and gaze aware VMC. In the VMC setting, gaze awareness was realised by careful alignment of video displays with respect to camera position, while the AMC setting used an early version of EyeCVE to replicate users’ gaze, head, and hand movements in their avatar embodiments. The goal of the exper- iment was to compare how people are observed to interact and behave when using these two visual telecommunication mediums. Gaze data, recorded from eye trackers worn by participants interacting with confederates in both systems, acts as the primary data source for evaluation and analysis. CA is coupled with the eye tracking data to assess participants’ ability to employ strategies commonly ob- served in collocated interaction, in order to successfully manage the mediated multiparty conversation. The theme of social agency is also addressed, assessing the extent to which participants engaged in AMC are observed to distribute their own gaze, and monitor others’ gaze, in a manner that is similar to when they are confronted with actual video representations of fellow interactants.

4.1

Experimental Aims and Expectations

The aim of this experiment was to investigate tracked gaze AMC and gaze aware VMC, in terms of how participants’ behave and use the capabilities of the systems in a multiparty conversational scenario. Of particular interest was the investigation of the role, and importance, of gaze as a communicational resource during both information exchange and management of the interaction. This included the ability to interpret attention from observing others’ gaze, signalling with gaze to select the next speaker, and establishing mutual gaze at critical moments of interaction. The basic hypothesis was that triads engaged in both AMC and VMC will be able to conduct conversation successfully, and will utilise gaze, measured by eye tracking, similarly in both mediums for purposes of both nonverbal expression and management of the unfolding interaction. Also, behaviour observed during the mediated communication is expected

4.2. Experimental Design 113