VR Toolkits

As with WIMP toolkits the most basic support available for development of virtual reality systems are collections of subroutines to be used by application programmers. Perhaps the most widely used of these at the present time is the Virtual Reality Mark-up Language (VRML) (Ames 1996) which consists of libraries of objects for building world models through the definition of geometry and object behaviour. Creating world models using VRML involves either writing them directly by hand or using a graphical modeller such as Cosmo Worlds (Cosmo 1999a). VRML is designed for deployment through a web enabled environment browser such as blaxxun (blaxxun

User Environment World Model Application

1999) or Cosmo Player (Cosmo 1999b) and as web-based VR is predominantly desktop VR, most interactions between the user and the VRML application occur through mouse and cursor key actions.

An alternative toolkit, Java3D (Sowizral 1997) is a collection of Java class libraries for building 3D applications. These classes provide the functionality for building 3D geometry with associated behaviours, as well as constructing the structures that are required to render the geometry. This equates to creation capabilities for both the environment and the world model. A variety of continuous input devices such as six-degrees-of-freedom (6DOF) trackers and joysticks are supported by Java3D (with access to keyboard and mouse events being provided by the standard Java API classes). Java 3D also defines an abstract class for dealing specifically with streaming input, which it collects as a time-stamped sequence of input values plus button/switch states. The advantage of Java3D over VRML from a developer’s point of view is that Java3D world models and environments can be easily integrated with applications built using standard Java classes.

Both Java3D and VRML were designed initially to support interaction between a single user and a single application. More sophisticated toolkits such as MASSIVE-2 (Greenhalgh 1999), DIVE (Carlsson 1993), SPLINE (Waters 1996) and NPSNET (Macedonia 1995), enable the prototyping and building of distributed multi-user interactive virtual environments. In DIVE for example, a number of application processes can exist within a single DIVE world model, each maintaining its own user interface of graphical objects. Once established, DIVE applications listen to events generated by users within the world. When an event specific to an application occurs, that application reacts according to some predefined control logic. Communication between DIVE entities takes place via shared data objects. It can be seen in Figure 4.9 that the core model of interaction is maintained, despite the numbers of users and applications.

Environment World Model Application

User User

User Application

Development of both DIVE and MASSIVE-2 applications involves programming in appropriate scripting languages (TCL in DIVE and a proprietary language for MASSIVE-2). More comprehensive tools are available which, like visual programming for 2D interfaces, enable the construction of 3D interfaces using environment builders with graphical drag and drop capabilities. Superscape is one such tool (Superscape 1999) which provides graphical world building capabilities over a proprietary file format called SVR. A toolkit with similar visual programming capabilities is MR Toolkit (University of Alberta 1999) which consists of set of subroutine libraries, support programs and a language for describing object geometry and behaviour. 3D environments are constructed out of an Object Modelling Language (OML) using a 3D modeller called JDCAD+. Support is provided in the form of appropriate device drivers for common VR input devices including a variety of 3D trackers, gloves and HMDs

The architecture of MR Toolkit, which is shown in Figure 4.10, consists of three layers of software. The bottom layer offers client/server support for input devices. The server continually samples the device and performs filtering on the received data. The client consists of a set of library routines that interface with the server and request data from it. The second software layer

processes data from devices and converts it into a format useful to an interface programmer (for example, the gesture recognition software associated with a dataglove could be embedded within this layer). The third layer provides services corresponding to initialization of all current input and output devices, as well as synchronization of data structures and display operations between

Level 1

Level 2

Level 3

Initialization Synchronization Packaged Services

Data Transfer Data Processing Conversion Gesture Recognition

Peripheral Devices Client

Server Client Server Client Server

Peripheral Devices Peripheral Devices

the two workstations required to provide binocular vision. MR Toolkit provides libraries of useful 3D interaction techniques as well as frameworks for producing new interaction techniques. The client/server model also makes it easy to introduce new devices by defining new device-specific client/server pairs.

VPL’s Reality Built for Two (RB2) is another system for rapidly prototyping virtual environments. It allows an interactive virtual environment to be created quickly without writing any code. A 3D modelling program called Swivel 3-D Professional is used to create 3D objects to inhabit the virtual environment. A central control system called Body Electric links objects within the virtual environment allowing behavioural and interaction rules to be defined. A single data array is fed integer values from each input device. This array is passed to a data flow network where scaling, offsetting and filtering can be performed on the integer values before they are passed to the graphical renderer where they become positional/ rotational values for objects and/or viewpoints in the world. Body Electric controls the flow of data from range of input devices to range of output devices defined within a VR application.

With RB2, a user constructs data flow networks using direct manipulation techniques on graphical representations of raw input, data manipulation modules, points and connecting wires. Raw input can come from any of the defined virtual devices and serve as the main source of data flowing into a data flow network. Data manipulation modules transform data using mathematical, logical and other operators. Points are the destinations of data leaving a network and have many attributes including position and orientation as well as associated constraints on the values of these attributes.

4.5.3 Discussion

The advantage MR Toolkit and RB2 have over systems such as DIVE and MASSIVE-2 is the ability they provide to create environments, objects, behaviours and interactions graphically using integrated tools. This is in contrast with toolkits consisting of code libraries that are only of use to experienced programmers. For example, the behaviour of objects within a DIVE environment requires programming knowledge of C and TCL/ Tk as well as DIVE itself. Building a world model including object behaviours in MASSIVE-2 requires knowledge of a proprietary scripting

Graphical 3D interface builder tools such as VRMog (Colebourne 1998) and AC3D (Colebourne 1999) can export graphical entities in DIVE, MASSIVE-2, VRML and Java3D file formats, but tend to do so inefficiently due to their generic nature. Generic graphical tools builders provide world-building capabilities for non-programmers, although addition of behaviours and constraints within the resulting world involves coding, which implies understanding of both the appropriate coding language and the toolkit-specific extensions to that language.

Despite these issues, what is clear from this review of VR toolkits is that their architectures adhere to a clear, conceptual model of interaction, in the same manner that WIMP development tools reflect aspects of Seeheim, MVC or PAC. In the case of VR systems the conceptual model is represented through an alternative set of high level components – the environment, the world model and the application. These high level components provide specific foci for different tools

which support the creation of one or more components of the model.