Cross reality, or XR, encompasses everything on the spectrum between fully virtual
environments, and fully real environments. This includes especially virtual reality (VR), augmented reality (AR), and augmented virtuality.
In this chapter we give a brief overview of existing technology and current devel- opments in the areas of virtual and augmented reality. We will explain the benefits of VR and AR, and outline associated challenges and opportunities offered, with an emphasis on biology and imaging. In the end we will sketch issues addressed in this thesis.
3.1
Virtual Reality, Augmented Reality, Mixed Reality
“Virtual Reality is the computer-generated simulation of a three-dimensional
image or environment that can be interacted with in a seemingly real or physical way by a person using special electronic equipment, such as a helmet with a screen inside or gloves fitted with sensors.” —Oxford Dictionary of English
With the termVirtual Reality we describe environments that simulate parts of the
real-world experience of human beings, such as the visual surroundings, auditory per-
ception, and sometimes even proprioception1in an interactive, computer-generated 1Proprioception is the sense of relative mo- tion and positioning of one’s own body and/or its parts.
three-dimensional environment. The world exterior to the simulated environment plays no role here, such that the user can become shut off from her real surroundings and fully immersed in the simulation, if it is convincing enough.
If the surroundings of the user are actually visible, e.g. via a set of glasses that are transparent and show the outside environment (or show them via cameras) and overlay information on top of it that extend or augment the capabilities or information content of the environment, we speak ofaugmented reality.
In the case of a mix of both, where there is a direct connection or overlap between the virtual, simulated world, and the real world, the setting is termedmixed reality. Mixed reality might take place anywhere in the virtuality continuum, except the
extremal points of fully real environments, or fully virtual environments, whilecross
26
Reality Augmented Virtuality
Reality AugmentedVirtuality Mixed Reality
…
Figure 3.1: Virtuality continuum according to [Milgram et al.,1995], where mixed real- ity encompasses all settings that are not the extremal points, and cross reality encloses the extremal points as well.
3.2
Historic Perspective — 1800s to 1990s
The first virtual reality “glasses” have been introduced in the 1850’s, as so-calledstereo- scopes, looking not unlike contemporary head-mounted displays. In the stereoscopes,
the user would insert a postcard that is split in the middle in two parts, showing the subject of the postcard from two slightly different perspectives corresponding the capturing an image with two eyes, as in the human visual system.
Figure 3.2: A Holmes-type stereoscopes to view left/right-eye images as single image. Public Domain.
In the early 1950, theSensorama was introduced, an immersive movie theater,
that not only included stereoscopic visuals, but also wind, sound, and even smell. The machine is shown in Figure3.3.
Figure 3.3: The sensorama. Image repro- duced from Sensorama, Inc. Advertisement, 1962.
With computer graphics still in it’s infancy, the first steps towards a head-mounted display mainly for military purposes, were made in 1968 by Ivan Sutherland [Suther- land,1968]. Sutherland developed a glasses-based virtual reality system (actually, augmented reality) that consisted of cathode-ray tubes mounted on the users head, with images being directed to the eyes by the means of mirrors. The tracking system for the contraption was suspended from the ceiling, looming over the user, hence the name of the system,The Sword of Damocles. The Sword of Damocles could display
wireframe models of geometric objects overlaid onto the user’s surroundings, and adapted to the viewpoint that had been calculated by the tracking system.
Figure 3.4: TheSword of Damocles. Note the cathode-ray tubes mounted to the sides of the user’s head, and the mirrors direct- ing the image to the eyes. Reproduced from [Sutherland,1968].
Big steps towards the current state of virtual and augmented reality were taken in the 1980s and 1990s by the University of Southern California’sMixed Reality Lab, and the company VPL, a spin-off of the lab. The lab developed not only head-
mounted displays, but full-body virtual reality suits, providing the user with a force- feedback system, and gloves developed for NASA that would react to virtual objects and the user’s grip [Zimmerman et al.,1987]. In addition to the personal systems based on head-mounted displays, room-scale systems such as theCAVE [Cruz-Neira et al.,1992] — a backronym forCAVE Automatic Virtual Environment — were
developed in the mid-1990s. In contrast to the HMDs, these systems use the tracking of the user not to display a perspective-corrected image on a screen attached to the user’s head, but on a (front or back-)projected wall or display at a distance to the user. Compared to HMDs, such CAVEs have the benefit that multiple people can use it simultaneously, with the constraint that only a single person will see the fully correct three-dimensional, immersive image. CAVE systems have found a large user base in the automative industry, and in architecture and design [DeFanti et al.,2010].
In the 1990s, interesting applications for various VR settings were explored in the research field. Especially UNC Chapel Hill’s Virtual Reality Lab created a lot