Piloting

Section5.2.2emphasised the importance of iterative prototyping before finalising an experimental de- sign. Similarly, the current experiment evolved over several stages. Initially, the design considered AMC only, and was inspired by Leal and Vrij’s [LV08] work investigating blinking during and after lying, and Lubow and Fein’s [LF96] work investigating pupillary size response to the “visual guilty knowledge” test. The original experimental protocol featured two conversational stages and one intermediary object- focused action, in which the participant performed tasks independently and out of the confederate’s sight. The first conversational stage involved the confederate providing training and information about the VR system. The critical object-focused period followed, during which participants either obeyed or disobeyed the confederate’s prior orders based on instructions provided by the experimenter. Fol- lowing this action, the second conversational stage began, which was also the critical period of data collection. During this stage, the participant was asked to recall the training information provided in the first conversational stage, and also their actions during the independent object-focused task, which was performed either truthfully or deceptively depending on their condition of obedience. Additionally, the visual guiltily knowledge test was performed, with the confederate showing various objects to the participant: some of which the participant would not have seen had they obeyed instructions during the object-focused stage. During this period of data collection, participants gaze, blinks, and pupil dilation was monitored for later analysis.

This experimental design, while incorporating physical movement in object-focused and conversa- tional interaction, proved problematic in terms of data collection. Regarding pupil size, the low ambient luminance of the CAVE caused participants’ pupils to remain in a dilated state, leading to limited varia- tion of pupil size response during the interactions. Also, blink detection was not robust, as the CAVE’s shutter glass frame forced the eye tracker’s viewing angle to be off-centre, thus leading to a permanently low pupil aspect ratio. The elements of free movement and object-focused action also precluded the possibility of comparison between AMC and VMC. Following the more exploratory nature of Chapter 4’s conversational experiment, further investigation into comparing AMC and VMC was deemed nec- essary for a full investigation into the issues raised when performing interaction mediated by virtual embodiments versus live video representations. Finally, the original design appeared to be too complex to investigate differences in truth telling and deception in AMC, and many participants expressed con- fusion during the experiment regarding their correct action. Therefore, E1’s final focused interpersonal experimental task was designed, piloted, and refined.

6.2.3

Apparatus

The experimental AMC and VMC was performed between two immersive display systems at UCL: the CAVE and the WALL. The two labs were located physically in adjacent rooms, allowing for collocated communication between the main experimenter and the confederate. For reasons relating to blink and pupil size tracking outlined in Section6.2.2, participants used the WALL, and the confederate used the CAVE. ViewPoint eye trackers were used in both AMC and VMC to capture gaze, blinks, and pupil size data.

6.2. Experiment 1: Interactions 158

Figure 6.1: EyeCVE supporting AMC between users of the WALL (left) and CAVE (right) immersive projection systems.

EyeCVE Setup

EyeCVE supported the AMC. Chapters4and5demonstrated the use of eye tracking to drive avatar gaze. The current study introduces blinking and pupil size as two additional tracked oculesic cues that are rep- resented by the advanced avatars. Implementation details and screen captures are presented in Sections 3.1.3and3.1.4. In the CAVE, EyeCVE operated as described in Section5.2.3, albeit a more mature ver- sion. The centralised EyeCVE server, maintaining the state of the virtual world and distributing changes to both clients, ran on the same machine driving the EyeCVE cluster displays. In the WALL, EyeCVE’s VE was projected on a single full HD (1920×1080 pixels) display wall with perspective-correct mono- scopic rendering. Head and hand tracking was achieved using a Polhemus Fastrak [Pol10], with sensors attached to a cap worn by participants as illustrated in Figure6.5. Audio communication between users of the CAVE and the WALL was established externally to EyeCVE using Google Talk [Goo10b]. The confederate wore a wireless microphone, while a desk-mounted microphone detected participant vocali- sation, aiming to reduce the number of devices required to be worn by the participant. Audio was output through speakers located in each lab. Figure6.1shows users engaged in AMC between both display systems.

The VE consisted of a meeting room populated with a rectangular table and two chairs. The absence of additional objects intended to eliminate the influence of distractions on participants’ gaze behaviour. The VE is presented in Figure6.2. Post-experimental analysis of gaze intended to investigate the propor- tion of time participants’ gaze was directed at the confederate, and the time spent looking away from the confederate. To aid this classification, a virtual grid, that was not rendered by EyeCVE, was positioned behind the confederate avatar as shown in Figure6.3. Section3.2.3presented a sample interaction anal- ysis performed on data captured in the current experiment. The grid aided classification of at/away gaze, together with direction of away-gaze according to logical reference by column and row.

Video Conferencing System

VMC was hosted between the CAVE’s front-wall and WALL displays. Bidirectional gaze awareness was achieved by aligning cameras and displays of remote users within Chen’s threshold [Che02]. Video

6.2. Experiment 1: Interactions 159

Figure 6.2: The VE in which the experimental AMC took place. The VE depicted a simple meeting room, with two chairs separated by a table. Before interaction, the virtual furniture was aligned to their real counterparts as illustrated in Figure6.1.

Figure 6.3: Grid (not rendered by EyeCVE) positioned behind confederate avatar to aid classification of participants’at and away gaze.

was streamed at full HD resolution in the WALL, and 1280×720 pixels in the CAVE using direct HDMI links between the camera located in one system and the projector display in the other. Users appeared life-size on both displays, and a plain black background behind the confederate intended to eliminate the influence of distractions on participants’ gaze behaviour. Audio communication was supported by Google Talk [Goo10b]. Figure6.4shows VMC as it appeared for the participant using the WALL.

Eye Tracking and Data Collection

Similar to Chapter5’s object-focused experiment, eye tracking was achieved using the head-mounted ViewPoint EyeTracker from Arrington Research. Eye trackers were mounted on two frames as shown in Figure6.5for use in either the WALL or CAVE. An additional reason for participants using the WALL was so that no stereo glasses, that may affect pupil size due to dark lenses, had to be worn. Hence, the eye tracker was mounted on an empty frame. In the CAVE, an eye tracker was mounted identically to as described in Section5.2.3. Both frames featured a camera which recorded the scene from the wearer’s perspective at a 150◦FOV. The wearer’s foveal fixation point was overlaid on the scene video, allowing for post-session analysis of action once synchronized with the separate audio stream. As documented in Section3.2, the eye tracker’s logging facility also served as the primary method for experimental data collection, with the tracked behaviour of users being streamed to the process, and output to a log file at 60 Hz. Data included gaze (2D X/Y coordinate, 3D hit-point in the VE, and fixation and saccade timings), blink signals, pupil size, speech signals, head and hand tracking, and markup data input by the experimenter.

6.2. Experiment 1: Interactions 160

Figure 6.4: Users engaged in VMC.

Figure 6.5: Eye tracker mounted on the WALL’s lens-free frame (left), and on the CAVE’s CrystalEyes 3 [Rea10] shutter glasses (right).

6.2.4

Population

A total of 24 participants with normal or corrected vision and no previous experience of telepresence systems were recruited from UCL’s campus through an advertising poster campaign. Participants were paid £10 to perform the study, which lasted for around one hour. Ages ranged from 19–62, with Eu- ropean, Asian and African origins were represented. While the effects of gender were not measured, both male and female participants took part, with 12 of each sex. Half of the participants performed the experimental interactions in AMC, and the remaining half in VMC. The two groups were balanced according to personality characteristics using the NEO Five-Factor Inventory questionnaire [CJM08], which was performed during the technical setup prior to the experimental interactions.

The experimental interactions consisted of six stages of questions, to which participants were in- structed to respond to with lies or with truths. Stages of questions (but not questions within a stage) and instructed response veracity were resequenced over participants to negate inter-experimental effects. The same male confederate questioned all participants in all AMC and VMC sessions, and was blind to participants’ current state of veracity. This aimed to minimise differences in confederate behaviour due to knowledge of how participants may be answering. Participants were aware of this, which in-

6.2. Experiment 1: Interactions 161 Table 6.1: Question stage and veracity condition presented to participants for each mediation type. Stages were divided intoGeneral 1 (G1), General 2 (G2), Recent 1 (Rc1), Recent 2 (Rc2), Remote 1 (Rm1), andRemote 2 (Rm2). Note that stages are resequenced by type (General, Recent, Remote), but not within stage class (i.e. G1 always precedes G2). Participants were instructed to tell the truth (T) or lie (L) during specific stages as shown.

Participant Question Stage and Veracity

1st T/L 2nd T/L 3rd T/L 4th T/L 5th T/L 6st T/L 1 G1 T G2 L Rc1 T Rc2 L Rm1 T Rm2 L 2 G1 L G2 T Rc1 L Rc2 T Rm1 L Rm2 T 3 G1 T G2 L Rm1 T Rm2 L Rc1 T Rc2 L 4 G1 L G2 T Rm1 L Rm2 T Rc1 L Rc2 T 5 Rc1 T Rc2 L G1 T G2 L Rm1 T Rm2 L 6 Rc1 L Rc2 T G1 L G2 T Rm1 L Rm2 L 7 Rc1 T Rc2 L Rm1 T Rm2 L G1 T G2 L 8 Rc1 L Rc2 T Rm1 L Rm2 T G1 L G2 T 9 Rm1 T Rm2 L G1 T G2 L Rc1 T Rc2 L 10 Rm1 L Rm2 T G1 L G2 T Rc1 L Rc2 T 11 Rm1 T Rm2 L Rc1 T Rc2 L G1 T G2 L 12 Rm1 L Rm2 T Rc1 L Rc2 T G1 L G2 T

tended to motivate them to lie plausibly and also to negate any inter-experimental effects in this regard. Participants were not informed of the significance of their oculesic behaviour. Finally, as described in Section6.2.5, setup procedures preceding the AMC and VMC were almost identical, aiming to provide a consistent experience to all participants prior to the experimental interaction. The condition sequences for all participants are presented in Table6.1.

6.2.5

Procedure

The experimenter at UCL was responsible for organising and running the experiments as well as local technical setup. The confederate using the CAVE was familiar with the technical architecture of the system, and was able to calibrate his body tracking devices independently. The experimental procedure consisted of seven major phases: briefing upon participant arrival; NEO Five-Factor Inventory question- naire; technical setup; eye tracker calibration; experimental interaction; Profile of Mood States (POMS) questionnaire [MLD92]; and finally a post-experimental interview.

Phase 1: Briefing

Upon arrival at the lab, participants were welcomed in a reception area, and were given an information sheet and ethical consent form. The information sheet set the scene for the forthcoming experiment involving a question-answer scenario, performed in AMC or VMC, with defined periods of truthful and deceptive response. Participants were left in private to fill out the forms, and were informed that they may ask any questions not directly relating to the experimental procedure or technical setup.