A study of techniques for selecting and positioning objects in immersive VEs: effects of distance, size, and visual feedback

A study of techniques for selecting and positioning objects in

immersive VEs: effects of distance, size, and visual feedback

Ivan Poupyrev

, Suzanne Weghorst

, Mark Billinghurst

, Tadao Ichikawa

IS Lab, Hiroshima University

HIT Lab, University of Washington

1-4-1 Kagamiyama

Box 352142

Higashi-Hiroshima-shi, 739 Japan

Seattle, WA, 98195 USA

(824)247669

(206) 685-3215

{poup, ichikawa}@isl.hiroshima-u.ac.jp

{grof, weghorst}@hitl.washington.edu

We present results of a formal evaluation of three direct manipulation interaction techniques for picking and posi-tioning objects in VEs: the “classical” virtual hand, ray-casting, and the Go-Go techniques. Our goal was to assess and compare the two most basic metaphors for selection and manipulation in VEs: virtual pointer and virtual hand. The main variables of interest were distance to and size of objects, interaction technique, and visual feedback. The results of the studies suggest that within the user-centered coordinate system, virtual pointing is an essentially two-dimensional metaphor, while the virtual hand is a three-dimensional metaphor. We also found that for some appli-cations the “classical” virtual hand technique appears to be obsolete and may be replaced by the ray or Go-Go tech-niques without a reduction in user performance. The paper reports these and other experimental results and discusses their implications for the design of VEs.

Keywords: virtual manipulation, virtual environments, 3D interaction techniques, evaluation, usability studies, ex-perimental testbed.

INTRODUCTION

With a rapid increase in the performance of high-end com-puter graphics systems as well as transition of 3D graphics on fast and inexpensive PC platforms virtual environment (VE) interfaces have become feasible enough to be used by practitioners in areas such as industrial design, data visuali-zation, training and others [1]. Development of useful VE applications, however, requires optimization of the most basic interactions, in particular object manipulation, so us-ers can concentrate on high-level tasks rather than on low level motor activities [2].

Currently, there is little understanding of how VE manipu-lation interfaces should be designed to maximize user per-formance in immersive environments [3]. Research that systematically investigates the human factors of immersive manipulation tasks, 3D devices, interaction metaphors and techniques, and their design implications remains sparse [2, 4, 5]; consequently VE designers have had to rely on their intuition and common sense, rather than on research results. However, as Professor Brooks has noted [6], “in watching many awful interfaces being designed ... I observed that the uninformed and untested intuition of the designer is almost always wrong.”

In this paper we present results of a formal experimental study that evaluates three direct manipulation interaction techniques for picking and positioning objects in VEs: the “classical” virtual hand, ray-casting, and the Go-Go inter-action techniques [3, 7]. The goal of the work is to assess and compare usability characteristics of two most basic 3D selection and manipulation metaphors: a virtual pointer metaphor that allows users to interact with objects by pointing at them, and a virtual hand metaphor that allows users to grab and manipulate objects with the virtual repre-sentation of their hand. Although object manipulation is among the most ubiquitous human-computer interactions in spatial 3D user interfaces, we are not aware of any formal studies that evaluate and categorize interaction techniques and metaphors for picking and positioning of objects in VEs. Prior research relates primarily to user performance as affected by various input and output devices and their char-acteristics [8, 9]. In contrast, the focus of this study is on the human factors characteristics of different mappings between user input, captured by input devices, and resulting actions in VEs - i.e., interaction techniques [3].

RELATED WORK

Object selection and positioning are among the most fun-damental interactions occurring between humans and envi-ronments, whether it is a “desktop” of 2D direct manipula-tion interface, 3D virtual environment, or the physical world [10, 11]. Prior research on manipulation in VEs, re-lates primarily to assessment of user performance as a function of input and display devices and their properties. For example, a pioneering study by Ware [12] demonstrat-ing applicability and ease of use of a 3D input device for a six degree of freedom (6DOF) placement task. A study by Zhai and Milgram [8] comparing isometric versus isotonic devices in various conditions of spatial manipulation, sug-gests that isometric input devices are better for rate control and isotonic for position control. Studies of stereoscopic versus monoscopic displays suggest that stereoscopy im-proves performance for complex manipulation tasks [13]. The effects of system performance characteristics (such as lag and frame rate) on user manipulation performance has also been studied [9, 14].

Investigation of the human factors of input and output de-vices has considerable value; however, the lack of system-atic research on manipulation interaction techniques, which map the user’s actions captured by input devices into

re-sulting actions in VE [3], can significantly limit their ap-propriate use in VE design. Interaction techniques essen-tially define the “look and feel” of VEs; a wide variety of techniques can be implemented using the same input de-vices and quite a few techniques for spatial manipulation have been demonstrated [7, 15, 16, 17]. Still, there have been few attempts to formally evaluate techniques for ma-nipulation in VE, assess their functional capabilities, and compare their relative strengths and weaknesses. A number of surveys have summarized and classified various ap-proaches for designing techniques for spatial input and identified problems and possible solutions [3, 18]. A formal study by Hinckley [19] evaluated and compared several spatial rotational techniques. More relevant here is the pio-neering usability study reported by Bowman and Hodges [11] that evaluated several VE techniques for manipulation at a distance. Although this study was somewhat informal and no quantitative data were collected, they provided use-ful preliminary observations of techniques.

Starting with early techniques that simply mapped position and orientation of the user’s hand onto position and orien-tation of manipulated objects [20], the field has been ex-panding with more sophisticated techniques such as flash light, aperture, Go-Go, World-In-Miniature technique [7, 11, 15, 17, 21] and others. This variety of techniques, how-ever, is also a source of difficulty. How do all these tech-niques relate to each other? Which interaction techtech-niques should be chosen for particular task conditions? Which among the parameters of interaction techniques, tasks, and environments should be considered to design efficient VE interfaces? These questions persist and merit careful scru-tiny by researchers and practitioners.

INTERACTION TECHNIQUES

Straightforward evaluation and comparison of manipulation techniques is difficult. There are a multitude of different techniques; their performance varies depending on the par-ticular implementation design; and studies of a parpar-ticular technique implementation may not be readily generalized to other implementations of the same technique, thus signifi-cantly limiting their external validity.

On the other hand, many techniques apparently relate to each other and share many common properties. For exam-ple, there are more similarities between ray-casting and flashlight techniques, than between ray-casting and tech-niques that use non-linear mappings to extend the user’s area of reach (as in Go-Go [7]). While evaluation of the ray-casting technique might provide insight into techniques similar to ray-casting, such as flashlight, it probably would not help in understanding techniques like the Go-Go. A taxonomy of techniques, classifying them according to their common properties, can be instrumental in understanding relations between techniques and directing their design and experimental evaluation.

A taxonomy of VE manipulation techniques

Analysis of current VE manipulation techniques suggests that most of them are based on a few interaction metaphors. Each of these basic metaphors forms the fundamental men-tal model of a technique and defines what users can do

(affordances) and what they can not do (constraints) when using the technique [22]. Particular techniques are essen-tially implementations of the basic metaphors, often ex-tending them in order to overcome some of the metaphor's shortcomings and constraints. For example, the flashlight technique enhances ray-casting by using a spotlight to ease selection of small objects [15]. These improvements often result in new constraints; for example, with the flashlight technique an ambiguity might occur if several small objects fall into the spotlight [17].

In Figure 1 we present a simple classification of VE ma-nipulation techniques according to their basic interaction metaphors. We divide the whole variety into exocentric and

egocentric techniques. Originated in studies of cockpit

dis-plays [23], these terms are used now to distinguish between two fundamental frames of reference for user interaction with VEs. With the exocentric interaction, also known as the God’s eye viewpoint, users interact with VEs from the outside (the outside-in world referenced display [23]). An example is the World-In-Miniature technique, which allows manipulation of objects by interacting with their represen-tations in a miniature model of the environment held by the user [21]. Although the exocentric techniques are interest-ing and important, their evaluation is outside the scope of this work.

With the egocentric interaction, which is the most common for immersive VEs, the user is interacting with VEs from inside the environment - i.e., the VE embeds the user [23]. Currently there are two metaphors for egocentric manipula-tion: virtual hand and virtual pointer [3, 11, 16]. With the

virtual hand, users can grab and position objects by

“touching” and “picking” them with a virtual representation of their real hand. A choice of input devices and mappings between real hand’s position and orientation and virtual hand’s position and orientation are some of the major de-sign factors that define particular techniques. For example a “classical” virtual hand technique provides one-to-one mapping between the real and virtual hands, while the Go-Go technique employs non-linear mapping functions to extend the user's area of reach [7].

Figure 1 Classification of VE manipulation techniques de-pending on their underlying metaphors.

With the virtual pointer metaphor, the user selects and ma-nipulates objects by pointing at them: when the vector ema-nating from the virtual pointer intersects with an object, it can be picked and manipulated [3]. The major design as-pects distinguishing techniques based on this metaphor are definition of virtual pointer direction, shape of the pointer (selection volume), and methods of disambiguating the ob-ject the user wants to select. In the simplest case, the direc-tion of the virtual pointer is defined by the orientadirec-tion of the user’s virtual hand, the virtual pointer is a “laser ray,” and no disambiguation is provided [24] (Figure 2). Some techniques define direction of virtual pointer direction us-ing two points: position of the user’s dominant eye and lo-cation of the tracker manipulated by the user [16, 17].

Interaction techniques to study

For this study we elect to evaluate those techniques that implement the basic egocentric metaphors, virtual hand and virtual pointer, as closely as possible. Investigation of the basic metaphors allows us to generalize our results beyond the specific implementation of the techniques and apply them to other techniques derived from investigated meta-phors. In this section we describe the implementations of techniques to be evaluated in this study.

Virtual pointer metaphor

We use the ray-casting technique for evaluation of the vir-tual pointer metaphor. Direction of the virvir-tual pointer is defined by orientation of the virtual hand. The working volume of the technique is an invisible infinite ray emanat-ing from the user’s hand (Figure 2); a short segment of the ray is attached to the hand to indicate the direction of pointing. To select an object, the user points at it and presses a button on the button device. Two variations of the technique has been evaluated: with and without visual feed-back. When visual feedback is applied, the color of objects changes when the virtual ray intersects with them.

Virtual hand metaphor

Two variations of the virtual hand metaphor were investi-gated: the classical virtual hand technique and the Go-Go interaction technique. With both techniques the user is pro-vided with the virtual hand, which position and orientation is controlled by the tracker attached to the user’s real hand (Figure 3). To select and pick a virtual object, the user in-tersects the object with the virtual hand and presses a button on the button device. While the virtual hand essentially simulates the way we manipulate objects in the real world (one-to-one mapping), the Go-Go technique uses a non-linear mapping to translate the measured distance to the user’s real hand into the controlled distance to the virtual one [7]. The non-linear mapping allows significant expan-sion of the user’s area of reach. Similarly to the virtual pointer, two variations of both hand techniques have been evaluated: with and without visual feedback. With the vis-ual feedback the object changes color when the virtvis-ual hand intersects with it.

THE EXPERIMENTS

Experiments that evaluated and compared three interaction techniques for selection and positioning objects in VEs were conducted within the framework of a Virtual Reality

Manipulation Assessment Testbed (VRMAT) [25]. The VRMAT is a tool that facilitates rapid design and imple-mentation of a variety of studies of immersive manipula-tion. It provides definitions of tasks and their properties, suggests experimental procedures including relevant inde-pendent and deinde-pendent variables, defines metrics and units for their measurements, and so on. In this paper we describe only those aspects of the VRMAT that are relevant to the studies.

Experimental tasks

Selection and positioning tasks were investigated in these studies. Experimental tasks required subjects to select or position virtual test objects (stimuli) using the interaction technique under investigation. Stimuli for the selection task are solitary test objects located in the user’s field of view (Figure 2). After being selected, the test object disappears, informing the subject that the task was completed.

The positioning task requires the subject to place a test ob-ject on top of a terminal obob-ject indicated by a different color (Figure 3). The positioning of the stimulus can be performed using iterative movements, i.e., subjects can pick, move, and release the test object several times until the task is accomplished. The task is accomplished when the test object is positioned on the terminal with the preci-sion specified a priori by the experimenter. The shapes for both test and terminal objects are cylinders with equal radii, providing a visual indicator of positional accuracy - the

Figure 2 Selection task: the user selects a solitary test ob-ject. The ray-casting technique is being evaluated.

Figure 3. Position task: the user puts a test object on top of the terminal object, indicated by a different color, using the Go-Go technique (the cube in the foreground represents the position of the subject’s physical hand).

better a test object is aligned on top of the terminal, the higher the task accuracy. After successful positioning, both objects disappear, cueing the subject that the task is fin-ished. The next test trial is then presented to the user.

Independent variables

The main independent variables of interest for the selection task were distance to the object, object size, interaction

techniques, and visual feedback. Objects' positions and

sizes are defined in a user-centered coordinate system similar to that used in Kennedy’s classic study of the reaching and grasping envelope of seated U.S. Air Force operators [26]. Position of a stimulus in VE is defined as the length d and orientation α, β of the vector pointing from the user's chest to the object (Figure 4). Distance d from user to stimulus is defined in terms of virtual cubits, a new unit of distance introduced in the VRMAT [25]. One virtual cubit is equivalent to the length of the user’s maximum reach (Figure 4); it is named after the cubit - a unit of measurement used in ancient Rome, equal to the distance between the elbow and the tip of middle finger. The ad-vantage of using virtual cubits is an ease of generalization of results from experimental studies to practical VE devel-opment: a stimulus located at a distance of one virtual cubit in the test environment would be located on the boundary of the user's reach for any user and any other VE independ-ently from the computational platform and software used. Virtual cubits also eliminate bias due to anthropometrical differences between subjects.

Size of the stimulus is defined as its non-occluded visual

size: the vertical and horizontal angles ϕ, φ the object occu-pies in the user’s field of view (Figure 4) [25]. Like virtual cubits, visual angles are also user-centered units: two ob-jects may have the same visual size, even if they are located at different distances and have different geometrical sizes. The benefit of visual angles is the separation of influence of distance and object size on user performance: if the object’s size is defined in terms of visual angles, then varying the distance to the object does not affect it’s visual size. Visual angles also allow for easy generalization of results beyond the particular test VE.

The main independent variables of interest for manipulation tasks were initial distance to the stimulus, distance to the

terminal position, required accuracy of positioning, and

interaction technique. Both initial and final distances are defined in termsof virtual cubits; required accuracy is de-fined as percentage of the terminal object being overlapped by the manipulated object. Higher percentage of overlap means higher required accuracy of positioning.

Performance criteria

Completion time, the time taken to successfully accomplish the tasks, is used as a primary performance criteria. For a selection task this is the time from the moment the stimulus appears until the moment it is successfully selected. For positioning tasks, completion time is measured from the moment the user picks a test object until the moment it is positioned with the required accuracy. Because position tasks allow iterative manipulation, we also measure the time of “net” manipulation, i.e., excluding the time required for each selection. Subjective criteria, such as subject satisfac-tion, is assessed through post-experimental questionnaires.

Subjects

Two groups of subjects were recruited from the laboratory subject pool. Ten males and three females served as a sub-jects for selection task experiments; eight males and four females served as subjects for positioning task experiments. Subjects ranged in age from 19 to 32; all subjects were right handed, as determined by the Edinburgh inventory.

Methods

A balanced within-subject (repeated measures) design was used for each task. Subjects were immersed in an environ-ment consisting of a ground plane and a virtual representa-tion of their hand. They wore a 6D tracking sensor on their dominant hand and held a button device (used for picking targeted objects) in the other hand. After donning the HMD subjects were asked to momentarily extend their tracked hand to its full natural reach for "virtual cubit" calibration. The environment then was re-calibrated according to the length of the virtual cubit.

Following a two-minute demonstration and explanation of the interaction techniques and test tasks, subjects had ap-proximately three minutes to practice tasks. During studies of the selection task each subject completed 18 sessions: six sessions for each interaction technique, three sessions with and three sessions without visual feedback. Fifteen condi-tions were defined in each session manipulating each of three different object sizes (4, 6 and 9 degrees) and five

Figure 4 Object position is defined as distance d and direction α, β in user-centered coordinate system. Object size is defined in terms of vertical (ϕ) and horizontal (φ) angles of the visual field subtended by the object.

different distances (0.7, 1, 2, 4 and 6 virtual cubits). Studies of the positioning task consisted of nine sessions: three ses-sions for each interaction technique. Nine conditions were defined for each session: four conditions for positioning at constant distances (0.8, 2.2, 3.5, and 6 virtual cubits), four conditions for positioning with changing distances to the object (from 0.8 to 1, from 1 to 0.8, from 3.5 to 6 and from 6 to 3.5 virtual cubits) and one condition for positioning at a constant distance (0.8 virtual cubits) with high required accuracy (90% of overlap). The rest of the conditions were defined with 80% accuracy.

The order of conditions presented in the experimental ses-sions was randomized to control for order effects. They were presented one after the other, with a four-second delay between them, until all conditions had been tested. In addi-tion to the on-line performance data, an informal quesaddi-tion- question-naire was administered after completion of the tasks to as-sess subjects' preferences and opinions.

Apparatus

The VRMAT testbed used for the experiments was imple-mented using a custom VR software toolkit developed as an extension of the Sense8 World Toolkit. An SGI Onyx RE2 workstation, equipped with a Virtual Research VR4 head-mounted display and Polhemus Fastrak 6DOF sensors, is currently used. A mouse is used as a button device for se-lection. The frame update rate is controlled at 15Hz.

Experimental results

Selection task

Figure 5 and Figure 7 summarize the effects of visual feed-back, distance and object size on selection time perform-ance while using the ray-casting technique. The box plots represent the distribution of the five subjects' scores around the median for each condition (collapsing over the orthogo-nal factor).

As shown in Figure 5, object selection time systematically increases with distance. This trend is supported by our ANOVA analysis (with visual feedback: F4,48 = 15.978,

p < 0.0001; without visual feedback: F4,48 = 23.869,

p < 0.0001). ANOVA reveals no treatment effect for the visual feedback at close and medium distances (0.7, 1 and 2 virtual cubits, F1,12 = 3.016, p < 0.108). Apparently, visual

feedback improves the user performance at far distances (4, 6 virtual cubits, F1,12 = 18.306, p < 0.001).

Collapsing over object distance (Figure 7) reveals the time it takes to select objects using ray-casting systematically decreases with object size (with visual feedback: F2,24 =

51.784, p < 0.0001; without feedback F2,24 = 30.688, p

<0.0001). ANOVA also revels a significant effect of visual feedback on user performance for all object sizes (F1,12 =

18.306, p < 0.0001).

Similarly, Figure 7 and Figure 8 summarize the effects of distance and object size on selection time performance

us-Figure 5 Box plots for selection times of objects located at various distances using ray-casting with and without visual feed-back (collapsed over object size).

Figure 6 Box plots for selection time of objects of different sizes using ray-casting with and without visual feedback (col-lapsed over object distance).

Figure 7 Box plots for selection time of objects located at various distances using the Go-Go with and without visual feed-back (collapsed over object size).

Figure 8 Box plots for selection time of objects of different sizes using the Go-Go with and without visual feedback (col-lapsed over object distance).

Figure 9 Mean selection times for objects at different distances using Go-Go and ray-casting techniques with visual feed-back applied (collapsed over object size).

Figure 10 Box plots for selection time of objects of different sizes using Go-Go and ray-casting techniques with visual feed-back (collapsed over object distance).

ing the Go-Go technique. As with ray-casting, collapsing over object size reveals a strong effect due to distance (with visual feedback: F4,48 = 28.253, p < 0.0001; without visual

feedback F4,48 = 35.681, p < 0.0001); collapsing over object

distance reveals a systematic increase in object selection time with size (with visual feedback: F2,24 = 22.701, p <

0.0001; without visual feedback: F2,24 = 48.761, p <

0.0001). ANOVA reveals no significant main effect of the visual feedback on user performance for the Go-Go tech-nique (F1,12 = 2.690, p<0.127).

Figure 9 compares mean selection performance times for ray casting and Go-Go interaction techniques for various object distances. Visual feedback data is used to compare both techniques on their peak performance. ANOVA re-veals that the ray-casting technique results in better per-formance on distances close to the user (F1,12 = 9.355, p <

0.01); with increased distance both Go-Go and ray casting provide similar performance (F1,12 = 0.008, p < 0.936).

Al-though the ray casting seems to be faster on far distances (6 virtual cubits), ANOVA reveals no treatment effect for technique at this distance (F1,12 = 0.948, p < 0.350).

Comparing selection times for two techniques across size conditions (Figure 10), two-tailed paired t-tests reveal sig-nificantly better performance for ray-casting in both the big and medium conditions (big: t = 4.541, df = 12, p < 0.001; medium: t = 3.109, df = 12, p = 0.009). In contrast, the Go-Go technique results in better selection performance for small objects (t=-3.026, df = 12, p<0.01).

Finally, we compared Go-Go and ray casting techniques with a traditional virtual hand technique for selection of objects of different sizes close to the user (0.7 virtual cu-bits). ANOVA reveals that the Go-Go technique results essentially in the same performance as the virtual hand (F1,12 = 0.22, p < 0.648) for all object sizes. Similarly,

ANOVA does not reveal performance differences between ray casting and virtual hand techniques for selection of small and medium size objects (4 and 6 degrees of visual field; F1,12 = 0.38, p < 0.849). However, ray casting results

in better performance for selection of large objects (9 de-grees of visual field; F2,24 = 7.96, p < 0.002).

Positioning task

Positioning objects from a close to a far distance and vice versa is difficult using the ray technique. The implementa-tion tested does not allow subjects to change the ray length, so they can position objects only through iterative move-ments. In pilot studies this method required an average of 10 iterations with a mean "net movement" time (i.e., with selection time subtracted) of 33.66 sec., compared to 4 it-erations and 11.89 sec. for the Go-Go.

However, the ray-casting technique can be efficient if ob-jects repositioning does not require changing their distance from the user. Figure 11 compares performance of Go-Go and ray-casting techniques for object positioning. ANOVA does not reveal performance differences between the ray casting and Go-Go for medium and far distances (F1,11 =

1.44, p < 0.711). Moreover, mean comparisons using the two-tailed paired t-test reveals better performance for ray-casting at a close distance (t=2.55, df = 11, p< 0.027).

Finally, we compared ray casting, Go-Go and virtual hand techniques for positioning occurring within the area of user reach (Figure 12). According to the box plot presentation, all three interaction techniques result in similar perform-ance in those conditions which require the user to move manipulated object closer or further (F1,11 = 1.359,

p < 0.28). When object positioning does not require changing the distance to the object, the ray casting and vir-tual hand techniques result in better performance than the Go-Go (F2,22 = 8.8, p < 0.002). Generally, the ray casting

technique results in poorer performance when a change in the distance is required (F2,22 = 17.786, p < 0.0001), while

Go-Go seems to be equally effective in all conditions (F2,22

= 1.309, p < 0.29). There is also a significant treatment effect of required accuracy on positioning performance (F1,11 = 103.243, p < 0.0001). An increase of required

posi-tioning accuracy, from 80% (lower accuracy) to 90% (high accuracy) target object overlap, results in a decrease of the performance for all techniques.

Subject’s comments

None of the subjects had difficulties in using either Go-Go, ray casting, or virtual hand techniques. The Go-Go tech-nique was rated as most enjoyable and intuitive, with ray casting second. This finding is supported by Bowman et al. [11]. Three subjects, however, preferred the plain hand, reporting that it was more familiar and simulated the way they interact in the real world. All subjects were dissatisfied Figure 11 Box plots for the positioning times of objects, when initial and final positions are at the same distance using the ray and Go-Go interaction techniques.

Figure 12 Box plots for the positioning times for virtual hand, Go-Go and ray casting techniques for conditions which require positioning without changing the distance, moving the object further, and bringing it closer.

with the performance of the ray casting at far distances and for selection of small objects, and all of them noted that visual feedback was helpful in these conditions. Some sub-jects noted difficulties in using the Go-Go at close distances - in particular, the distance where linear mapping switches to non-linear. Subjects reported that one of the main sources of difficulty in positioning objects at a distance was limited visual cues, rather than shortcomings of the tech-niques themselves. Subjects simply could not see if the ob-ject was being positioned correctly.

Discussion

Our findings for ray-casting performance suggest that it is essentially a two-dimensional technique, defined in terms of the user-centered distance/pitch/yaw coordinates used in this study, rather than world-centered Cartesian (x/y/z) co-ordinates. For object selection within the user field of view and for repositioning of objects at a constant distance from the user, ray casting is an efficient and effective technique. Ray casting is far less useful as a repositioning technique when change in distance is required. Indeed, even within the area of user reach, ray-casting performs better for posi-tioning at constant distance. The Go-Go, on the other hand, resulted in essentially the same performance for all those conditions.

Adding visual feedback does not necessarily improve user performance. For selection of solitary objects located rela-tively close to the user, ray casting provides essentially the same performance with or without visual feedback. The Go-Go technique resulted in essentially the same perform-ance for most of the task conditions with or without visual feedback. This could be due to the fact that with techniques based on the virtual hand metaphor, the user can see when the hand intersects the object; thus visual feedback is an inherent part of the technique. Visual feedback, however, improves the user's performance for boundary conditions -for example, in selecting small objects located far away using the ray-casting technique. Also, under certain condi-tions such as selection of occluded objects or objects within a group, enhancing the techniques with visual feedback might improve user performance.

For both interaction techniques we see that as object size decreases, the "target" object is increasingly harder to "hit." This finding is consistent with expectations and appears to represent a "Fitts Law" phenomenon. Similarly to Go-Go, ray-casting selection exhibits a performance falloff due to object distance. Reports by subjects suggest that the de-crease in performance of ray casting with inde-creased distance may be due to difficulties with hand-eye coordination and tracker noise. With ray-casting, the influence of distance on user performance decreases with the increase of object size; for a large object the selection time is essentially the same in all tested distances.

Overall, both ray-casting and Go-Go seem to provide es-sentially similar performance for selection at medium dis-tance; while ray casting is more effective at close distances and in selection of big objects, and Go-Go is more effective in selection of small objects. For the positioning task both techniques seem to results in the same performance when

the task does not require a change in distance; however, the Go-Go technique is superior for those tasks requiring changes in distance to object. Other aspects of the manipu-lation task may interact with these main effects. Object oc-clusion and density, for instance, may differentially affect interaction performance with the two techniques. The higher performance of the Go-Go for selection of small objects can also be an advantage for selection of partially occluded objects, due to their diminished visual size. Finally, both ray-casting and Go-Go techniques provided essentially the same or better performance than a classical virtual hand interaction for the conditions of immersive manipulation selected for these studies.

CONCLUSION

Our findings in this study are consistent with the notion that selection and positioning of solitary objects can be either a 2D or 3D task, depending on whether the object distance is manipulated. Within the user-centered coordinate system used in this studies, the virtual pointer seems to be essen-tially a 2D manipulation metaphor, while the virtual hand, seems to be a 3D manipulation metaphor. The 2D nature of the ray technique implies that we may be able to apply well-developed guidelines and techniques from 2D graphical user interface design for development of immersive ray-based interaction dialogs. Furthermore, the “classical” vir-tual hand technique appears to be obsolete and may be re-placed by the ray and Go-Go techniques without a reduc-tion in user performance. By affording manipulareduc-tion at a distance, both the ray and Go-Go techniques provide more functionality relative to the plain virtual hand.

The results of the study suggest that each technique pro-vides its best performance within a certain area of effective manipulation and for certain object sizes. Improvements to the techniques, such as introduction of visual feedback, do not effect user performance for these "standard" conditions, but rather extend the limits by improving user performance at the boundary conditions (Figure 5). Therefore, develop-ment of better manipulation techniques, which would allow for effective manipulation at further distances and for smaller object sizes, may not be the only way to build effi-cient VE interfaces. Instead of improving interaction tech-niques, developers can take another route: improving spa-tial design of virtual environments to allow the existing techniques their best performance. For example, the ray casting technique can provide a satisfactory performance even without visual feedback if objects are located within 3 virtual cubits or have sufficient visual size (more than 4 degrees of visual fields). If these conditions are satisfied, the most generic implementation of ray-casting would per-form well, resulting in simpler user interfaces.

Certainly for some applications it is not possible to design the VE around the techniques. In this case alternative ap-proaches can be investigated, such as combinations of fly-ing and manipulation or applications of the exocentric tech-niques. Nevertheless, there are many application domains where designers do have freedom to fit the environment to the interface - for example, VEs for information visualiza-tion VEs.

The growing acceptance of VE technology will require more attention to optimizing the immersive interaction in order to maximize user performance. The research reported here is just a small step toward understanding the human factors behind manipulation in VEs and their design impli-cations. Future studies of VE manipulation should further investigate the design aspects of the techniques and their influence on user performance; assess usability of the tech-niques in other conditions of manipulation tasks; evaluate combinations of manipulation techniques with navigation techniques; and explore possible ways to integrate various techniques into seamless and intuitive interaction dialogues.

ACKNOWLEDGMENTS

This research was partially sponsored by the Air Force Of-fice of Scientific Research (contract #92-NL-225) and a grant from the HIT Lab Virtual Worlds Consortium. The authors want to especially thank Edward Miller for his comments and suggestions as well as his help with the VRMAT development. We would also like to thank Sisinio Baldis, Jennifer Feyma, Prof. Masahito Hirakawa, Jerry Prothero, Atsuo Yoshitaka, and all subjects who partici-pated in the experiments.

REFERENCES

1. Göbel, M., Industrial applications of VEs. IEEE Com-puter Graphics&Applications, 1996. 16(1): pp. 10-13. 2. Stanney, K., Realizing the full potential of virtual

real-ity: human factors issues that could stand in the way. Proceedings of VRAIS'95. 1995. ACM. pp. 28-34. 3. Mine, M., Virtual environment interaction techniques.

UNC Chapel Hill CS Dept.: TR95-018. 1995.

4. Herndon, K., A. van Dam, and M. Gleicher, The chal-lenges of 3D interaction: a CHI'94 workshop. SIGCHI Bulletin, 1994. 26(4): pp. 36-43.

5. Durlach, N. and A. Mavor, eds. Virtual reality: scientific and technological challenges. . 1995, National Academy Press: WA. 542.

6. Brooks, F.P., Grasping reality through illusion - interac-tive graphics serving science. Proceedings of CHI'88. 1988. ACM. pp. 1-11.

7. Poupyrev, I., M. Billinghurst, S. Weghorst, and T. Ichi-kawa, Go-Go Interaction Technique: Non-Linear Map-ping for Direct Manipulation in VR. Proceedings of UIST'96. 1996. ACM. pp. 79-80.

8. Zhai, S. and P. Milgram, Human performance evalua-tion of manipulaevalua-tion schemes in virtual environments. Proceedings of VRAIS'93. 1993. IEEE. pp. 155-61. 9. Watson, B., V. Spaulding, N. Walker, and W. Ribarsky,

Evaluation of the effects of frame time variation on VR task performance. Proceedings of VRAIS'96. 1996. IEEE. pp. 38-52.

10. Foley, D., V. Wallace, and V. Chan, The human factors of computer graphics interaction techniques. IEEE Computer Graphics & Applications, 1984(4): pp. 13-48. 11. Bowman, D. and L. Hodges, An evaluation of

tech-niques for grabbing and manipulating remote objects in

immersive virtual environments. Proc. of Symposium on Interactive 3D Graphics. 1997. ACM. pp. 35-38. 12. Ware, C., Using hand for virtual object placement.

Vis-ual Comp., 1990. 5(6): pp. 245-253.

13. Spain, E. and K. Holzhauzen, Stereoscopic versus or-thogonal view displays for performance of a remote ma-nipulation task. Proceedings of Stereoscopic Displays and Applications II. 1991. SPIE. pp. 103-110.

14. MacKenzie, I. and C. Ware, Lag as a determinant of human performance on interactive systems. Proceedings of INTERCHI'93. 1993. ACM. pp. 488-493.

15. Liang, J., JDCAD: A Highly Interactive 3D Modeling System. Computers and Graphics, 1994. 18(4): pp. 499-506.

16. Pierce, J., A. Forsberg, M. Conway, S. Hong, R. Ze-leznik, and M. Mine, Image plane interaction techniques in 3D immersive environments. Proceedings of Sympo-sium on Interactive 3D Graphics. 1997. ACM.

17. Forsberg, A., K. Herndon, and R. Zeleznik, Aperture based selection for immersive virtual environment. Pro-ceedings of UIST'96. 1996. ACM. pp. 95-96.

18. Hinckley, K., R. Pausch, J. Goble, and N. Kassell, A survey of design issues in spatial input. Proceedings of UIST `94. 1994. ACM. pp. 213-22.

19. Hinckley, K., J. Tullio, R. Pausch, D. Proffitt, and N. Kassel, Usability analysis of 3D rotation techniques. Proceedings of ACM UIST'97. 1997.

20. Ware, C. and D.R. Jessome, Using the bat: a six-dimensional mouse for object placement. IEEE Com-puter Graphics&Applications, 1988. 8(6): pp. 65-70. 21. Stoakley, R., M. Conway, and R. Pausch, Virtual reality

on a WIM: interactive worlds in miniature. Proceedings of CHI'95. 1995. pp. 265-272.

22. Erickson, T., Working with interface metaphors, in Readings in human-computer interaction: toward the year 2000, R. Baecker, J. Grudin, and W. Buxton, Edi-tors. 1995, Morgan Kaufmann Publishers, Inc.: San-Francisco, CA. pp. 147-151.

23. Wickens, C.D. and P. Baker, Cognitive Issues in Virtual Reality, in Virtual Environments and Advanced Inter-face Design, T.A. Furness and W. Barfield, Editors. 1995, Oxford University Press: NY. pp. 514-542. 24. Jacoby, R., M. Ferneau, and J. Humphries, Gestural

Interaction in a Virtual Environment. Proc. of Stereo-scopic Display and VR Systems: The Engineering Real-ity of Virtual RealReal-ity. 1994. SPIE. pp. 355-364.

25. Poupyrev, I., S. Weghorst, M. Billinghurst, and T. Ichi-kawa, A framework and testbed for studying manipula-tion technique for immersive VR. To be presented at the ACM VRST'97. 1997.

26. Kennedy, K., Reach capability of the USAF population: Phase 1. The outer boundaries of grasping-reach enve-lopes for the short-sleeved, seated operator. USAF, AMRL: TDR 64-56. 1964.