Thus far, I have discussed mapping a variety of virtual objects onto a single real object. It is important to note that warped spaces can be used also to map a variety of real objects onto a single virtual object; all that is required is some real-virtual mapping. This idea may enable VE designers to quickly set up passive haptics in coarse locations to provide haptic feedback, instead of precisely measuring and placing passive haptics. I implemented a proof of concept of this idea.
I used the same system as before and placed a low-cost two-faced 202x302foam board on the
square table (Figure 4.10). It was quickly taped to the table in an arbitrary position with an arbitrary angle between the two faces. The two board faces turned out to be close to but not quite the same size (widths ~14.252 and ~15.752).
4.5.1 Finger tracking and calibration
Every user’s index finger is different and the system must calibrate for these differences. The user’s finger is placed flat at a known location on a surface (Figure 4.11). The difference between the tracker data and the known location is computed, and this difference is subsequently used to transform the tracker data so it accurately represents the user’s fingertip location.
Figure 4.10: A low-cost two-faced passive-haptic foam board
Figure 4.11: Finger calibration
4.5.2 Determination of physical geometry
The system needs to know the (arbitrarily placed) foam board’s geometry before a virtual object can be mapped onto it. In the current system, the user points, with a tracked finger, to each corner of the physical object (in the order specified in Figure 4.10). The system linearly interpolates these points to generate vertices for the physical geometry. This technique works for physical objects consisting of planar facets.
The generated vertices are already in the coordinate space of the finger’s tracking system, so when the user touches a corner, no additional transformation (beyond the space warp) is required. If there are systematic errors in the tracking system, then sampling the physical surface with more data points should still enable the system to generate a space warp that is valid for the tracking system’s coordinate space.
Once the physical geometry is captured, correspondences between points on its surface and predetermined points on the virtual surface are passed to the space warping system.
4.5.3 Discrepant objects
I mapped a variety of virtual objects onto the foam board. Three are shown in Figure 4.12. The initial implementation did not warp finger orientation. However, not having orientation warping introduced an unintended tactile discrepancy: the forces felt on the real fingernail did not match what was seen in the VE. For example, when a real board face was angled and a virtual board face was not, the real fingernail would experience a force near the side, rather than near the center as the VE would indicate. I therefore subsequently warped hand orientation as well. The entire process of determining geometry and warping space for a newly placed board took less than one minute.
As in earlier explorations, interacting with these discrepant objects was plausible. The curved virtual board in the upper right of Figure 4.12 felt odd near the real board’s crease, but was otherwise effective.
Figure 4.12: A user (wearing an NVIS nVisor SX60 HMD) touches a corner of the physical foam board. The virtual finger position is warped to touch the corners of three different virtual objects. The insets show the distances between the virtual finger and the superimposed real finger (red).
CHAPTER 5
Redirected Touching: Task Performance
1
In Chapter 4, I introduced Redirected Touching, a perception-based technique for mapping many differently shaped virtual objects onto a single real object and vice versa. Redirected Touching warps virtual space and introduces real-virtual hand-motion discrepancies to accommodate real-virtual object discrepancies. While informally exploring many discrepant objects, I quickly discovered that the set of research questions is large and multidimensional. There are many kinds of discrepancies and many ways to warp spaces. Systematic investigation required a guiding application.
Military aircraft pilots and maintenance crews must learn to perform cockpit procedures, such as the sequences of buttons and switches required for aircraft start-up, shut-down, and emergencies. Real aircraft and full flight simulators can be used to train these skills, but can cost hundreds of dollars or more per hour [Vincenzi et al., 2008]. Full simulators are unavailable in deployed settings [Andre et al., 2004], and for many procedures they are not required; low-fidelity trainers with mock cockpits can effectively train cockpit procedures [Prophet and Boyd, 1970].
Figure 5.1: Lockheed Martin® C-130 Hercules® cockpit 1Portions of this chapter were previously published elsewhere [Kohli et al., 2012].
Cockpits are complex (Figure 5.1), and one of the challenges trainers face is training “muscle memory” and spatial knowledge [Degani and Wiener, 1993; Baseops.Net, 2012]. Pilots must use checklists for many procedures, but they must know the relative locations of cockpit controls so that they can be accessed efficiently when needed. Mock cockpits enable trainees to reach for cockpit controls. However, they are designed for a specific aircraft. For a new aircraft model, another mockup must be built, or the existing one must be changed [Wurpts, 2009]. Reconfigurable trainers do exist, but they typically support reconfiguration by repositioning or swapping out components [Schiefele et al., 1998; Williams et al., 2004; Lockheed Martin, 2012]. Instead, Redirected Touching may enable a single quickly set-up physical mockup to represent many virtual cockpits, eliminating the need to change the mockup for each new aircraft.