Integral Attribute Manipulation with Separable Input

Intuitively, 3D object transport seems to be most compatible with integral 3D trans- lation input. However, other mappings may also be suitable. Balakrishnan et al., for example, proposed design adaptations to the common mouse to better support 3D interaction [21]. The underside of their Rockin’ Mouse was rounded such that the device afforded tilting. Tilting about the depth axis was mapped to translation along the vertical axis, while the common two-axis translation of the mouse was used to control virtual object motion on a ground plane. This design enabled the simultane- ous operation of translation across 3D space – although with two different types of movement. They demonstrated that 3D positioning could be performed up to 40% faster with the Rockin’ Mouse compared to a common 2D mouse with mode switch- ing for additional degrees of freedom. An analysis of motion paths revealed that 49% of the motion input with the Rockin’ Mouse involved all three dimensions si- multaneously. Simultaneous translations along the three axes occurred primarily in the ballistic phase, in which users achieve coarse approximation. During the subse- quent closed-loop phase, in which more fine-grained adjustments were performed, a tendency for axis separation was observed.

This example is another demonstration that rotation and translation can be operated simultaneously – at least at the motor level. An unusual mapping enabled the integral operation of 3D translation without lifting the arm. This approach seems to compro- mise stimulus-response compatibility, but fortunately, our mental models are flexible and can adopt to changing configurations. Research on stimulus-response compat- ibility shows that some non-isomorphous mappings are more comprehensible than others [275, Chapter 9]. Balakrishnan et al. mapped clockwise tilting of the mouse to upwards motion, while counterclockwise tilting moved the cursor down. With the mouse located to the right of the display, this mapping corresponds to Warrick’s principle [275, Chapter 9.2]: It describes the perceptional preference for mappings from rotation input to translation where the linear motion direction corresponds to the closest tangent of the rotary controller. Corresponding examples are known from rotary knobs for the tuning of old-fashioned radio receivers. In case of the Rockin’ Mouse (in right-hand operation), the left side of the device is closest to the display

and adequately moves up and down during rotation (Figure 3.3). We can thus per- ceive a coherent mapping from 3D position input to 3D position control if we consider the position of the left side of the device as the input coordinate.

Figure 3.3: The Rockin’ Mouse of Balakrishnan et al. [21] enables full 3D trans-

lation with the device comfortably resting on the desktop surface. Tilting the mouse moves a 3D cursor (sphere with wireframe bounding box) up and down. The applied mapping corresponds to Warrick’s principle [275, Chapter 9.2]. The experimental task of Balakrishnan et al.’s study was to place the 3D cursor in- side a one third larger translucent target cube (in the upper left corner of the screen in this illustration).

The question remains, whether this input mapping enables efficient manipulation of the three degrees of freedom. Balakrishnan et al. compared 3D object positioning with the device to a mouse-based technique and reported a performance advantage of approximately 30%. Not much detail on the task was provided, but from the de- scription of the setup with a visual interaction volume of approximately 30 cm2_{, we} can derive a maximum index of difficulty of about 5. They recorded mean task com- pletion times of 5.5 seconds for this task after training over five blocks of task rep- etitions. This would correspond to a very low thoughput of less than 1 bit/s. The report is not clear whether the task required spatial selection of the manipulation ob- ject in the Rockin’ Mouse condition. If this was the case, then the task would have consisted of two aimed movements in 3D space with a combined index of difficulty of 10 bits at maximum. The throughput would then correspond to approximately 1.8 bits/s, which is still slightly less than the average throughput rate for 3D pointing tasks with unconstrained 3D motion input (see Chapter 2). The Rockin’ Mouse does not seem to support the known performance of 2D pointing with the mouse in 3D pointing tasks.

Integrality and Separability in Manipulation Tasks 39 In the 2D mouse condition, participants had to select one of three visible faces of a cube and then move it along the corresponding geometric plane. Therefore, in the mouse condition, the task consisted of four subsequent aimed movements, each with two degrees of freedom. Moreover, since the bounding box of the manipulation ob- ject was aligned with the screen plane, the perspectively distorted faces that enabled motion in depth offered only a small target width. The combined index of difficulty of the four subtasks in the mouse condition can be estimated to be in the range of 15- 16 bits. Considering average throughput rates in 2D pointing tasks (about 4.5 bits/s), movement times should not exceed 3-4 seconds. Balakrishnan et al. recorded average task completion times of about 7.5 seconds after training. Apparently, the switching from one subtask to the next was also time consuming. The example of the Rockin’ Mouse, therefore, demonstrates that the integral manipulation of degrees of freedom can be beneficial, despite the required coordination overhead. The elimination of mode switching allows users to perform integral movements in a coherent action. In this particular experiment, however, the mouse could have performed much better if the motion direction was toggled with mode keys.

Another notable example of 3D placement with separable input is the Balloon selec- tion technique presented by Benko and Feiner [29]. They suggested to use multitouch input for the placement of a 3D cursor following the metaphor of a floating balloon on a cord (Figure 3.4). The x/y position is controlled with one hand moving along a touch surface. The distance between two fingers of that hand defines the size of the cursor. Movement perpendicular to the touch surface can be controlled with an- other hand inducing further touch input relative to the first one. They compared this technique to direct 3D pointing with a 3D wand and a keyboard technique where movement along all three axes was manipulated completely separate with discrete keystrokes.

Figure 3.4: Benko and Feiner suggested Balloon Selection. A multitouch input

technique to control the position and size of a 3D cursor. One hand controls the x/y position of a 3D cursor above the touch surface. The distance between two fingers of that hand defines the size of the cursor. Additional touch input by an- other hand controls the height of the cursor by adjusting the distance between both hands.

The task description, allows us to estimate an index of difficulty in the range of 3-5 bits. The average task completion times for the 3D wand and the Balloon technique were in the range of 6-6.5 seconds, while it took about 12 seconds on average with the keyboard technique. Considering a typical error rate of about 4% the throughput of the two faster techniques would be in the range of 0.75 bits/s. This is not neces- sarily a convincing performance, but in fact, only in the keyboard condition, users achieved an error rate of 4.1%. With the balloon technique it was slightly higher at 5.5%, but with the 3D wand, the error rate increased to 16.1%. This means that the effective throughput with the 3D wand technique was much lower than it was with the multitouch technique.

One reason for the overall performance below average may be the limited perceptual quality of the output device used in the study. The stimuli were presented on an op- tical see-through head-mounted display with a resolution of 800 x 600 pixel. A more plausible reason, however, seems to be the required accuracy. For target volumes of 10, 8, and 6 mm3_{, the authors measured average error rates across devices above 6%.}

For the smallest target size of 4 mm3it went up to about 14%, which indicates, that Fitts’s Law might not hold any more, because the target size was below the physically achievable accuracy. Moreover, the experiments involved additional adjustments of the cursor size, which added variability on yet another separate dimension.

In any case, the comparison of the Balloon technique with integral 3D motion input indicates a competitive edge of separable over integral input for 3D aimed move- ments. The reason for this advantage is perhaps the physical support provided by the multitouch sensor, which facilitates accurate placements. The distribution of con- trol between both hands also seems to be advantageous in this case. The cooperative bimanual 3D motion control supports simultaneous as well as separate adjustments of the involved degrees of freedom. In the following section we will discuss further examples of input combinations with multitouch interfaces.