The digital virtual

2.7 Sutherland’s Head Mounted Display

2.8 A Link Jnr Combination Training device

Effectively the digital ‘virtual’ enters into visual culture with early experimentation in human–computer interface design; the means by which a human interacts with the machine. Early forms of ‘VR’ offered to provide an interface that removes all signs of a mediating appa- ratus between the user and computer generated or stored image, information or content.

It was seen as promising to dispense with the residual forms of the computer screen, keyboard, and mouse (hang-overs from television, typewriters and mechanical controls). As the 1960s pioneer of graphic and immersive computer interfaces Ivan Sutherland put it, we should ‘break the glass and go inside the machine’ (quoted in Hillis 1996), or, in the words of the more recent developer of VR systems, Jaron Lanier: in VR ‘the technology goes away’ because ‘we are inside it’ (quoted in Penny 1995: 237).

Ivan Sutherland was a key figure in the operational and conceptual history of VR, and a pioneer of computer graphics and simulation technologies, who worked within militarily funded research programmes. In this context, Sutherland tackled the question of what sym- bolic form a computer’s output might take or, as we would now put it, what would be the form of the human–computer interface? Given that a computer’s internal activity is a vast and continuous stream of electrical impulses, Sutherland asked how the results of this invisible activity might be ‘output’ or externalised. What form – language or sign system – should be used to display the results of computation? Sutherland demonstrated that these impulses could be translated into an electron beam that was visible on a visual display unit – a screen. The origin of contemporary computer graphic interfaces, such as those used by the Apple operating systems or Microsoft Windows, is first seen in his now famous prototype ‘Sketchpad’.

Sutherland also envisaged the possibility of going beyond graphic display to make the results of computation tangible. He conceived that if a computer reduced and processed any kind of information as a series of impulses, given the appropriate algorithms and program- ming, the physical movement of the human body – and even material resistance to that movement – could also be encoded as information which the computer could process.

From imitation to simulation

Sutherland’s inspiration was the joystick of a Link Flight Trainer in which ‘the feel’ of a mocked-up aircraft’s parts, moving as if against wind and air pressure, was mechanically fed back to the trainee pilot. In working upon the development of flight simulators, Sutherland drew upon several breakthroughs in technology and mathematics (see Woolley 1992: 42–48). Sutherland’s work showed how human actions could become computable information that was then passed back to the human subject, via servo mechanisms and sensors, to then inform or control their further actions. This took a graphic and tactile form in a cybernetic ‘feedback loop’ between computer and human being (see 5.1).

Where Sutherland’s inspiration makes empirical references to a real aeroplane by a func- tionally quite unnecessary copying of its wings and tailplane, after Sutherland the flight simulator eventually becomes an enclosed environment, a ‘black box’, with no external, mor- phological reference to aeroplanes at all. Yet once such a ‘black box’ is entered the sensory conditions experienced in real flight can be more fully generated to include, for instance, the programmed vicissitudes of the weather, or engine failure, acting upon the virtual aircraft. Such simulators, without any external mimetic reference to real planes, can then simulate planes that have not yet been built or flights that have not yet been taken. Let alone there being no imitation of wings or tailfins as in the Link Trainer, there are no particular planes to imitate. Here we meet the distinction between imitation and simulation: the notion that in sim- ulation (as against imitation or mimesis) the model now, in some senses, precedes the

See Bolter and Grusin (1999: 161–167) for a brief discussion of VR in these terms or as ‘the end of mediation’

From the end of the Second World War, the US government began serious funding of research aimed at improving flight simulation and the computation of ballistic tables, the calculation of the trajectory of shells and missiles necessary to accurate targeting. The great cost of modern military aircraft, and the enormous demand for ballistic calculation, fuelled the development of electronic/digital computation. This was not the first time that the demand for calculation threatened to outstrip the human capacity to produce tabulated data fast enough and then drove the development of computers. See Mayer (1999: 506) on Babbage’s Difference Engine (a version of which was completed in 1854), a mechanical computer which was partly a response to the demands for maritime navigation in the nineteenth century. Woolley (1992: 49) reports that in the 1940s the 60-second trajectory of a single missile would take twenty hours to work out by hand. One of the first electronic mainframe computers, the ENIAC (1944) took 30 seconds. For more on the military origins of cybernetics, and therefore contemporary computing, see Part 5

reality – a reversal of the expectation that ‘models’ are built (imitate) pre-existing realities. (See Woolley 1992: 42–44 for a more detailed discussion.)

This is a distinction that can be hard to grasp. For present purposes we will be content with the following recognition: what distinguishes simulation from imitation is that an artefact that is a simulation (rather than a copy) can be experienced as if it were real, even when no corresponding thing exists outside of the simulation itself. We are, after all, now familiar with such simulated reality effects from watching the seamless insertion of computer animations and special effects in contemporary blockbuster movies and television adverts. (For more on ‘simulation’, see 1.2.6, and later in this section.)

‘A head-mounted three dimensional display’

In a 1968 scientific paper of this name, Sutherland reported on an apparatus that would, in effect, generalise the flight simulator. Here, Sutherland made a conceptual move similar to that made by Alan Turing when he conceived of the computer as a ‘universal’ machine. Sutherland built an apparatus that included a rudimentary head-mounted display. The HMD’s basic purpose was to ‘present the user with a perspective image which changes as he moves’ (Sutherland 1968: 757). The space that the wearer of the helmet ‘saw’, and which shifted as they moved their head, was generated mathematically. It was structured by a three- dimensional Cartesian grid with its three spatial co-ordinates imaged stereoscopically on the binocular TV screens held close before their eyes.

For Sutherland, this apparatus had no specific purpose such as flight simulation. It was a visual and tactile interface with a computer, an alternative to the early punch cards, or to a keyboard, light pen and screen. Instead of human–computer interfaces being holes punched in paper tape or two-dimensional manipulable graphics displayed on a VDU, this interface was, however rudimentary, spatial, visual, tactile and kinaesthetic. An important element in the history of Western visual culture makes an appearance in our brief account of Sutherland’s work; a conception of space which is historically and culturally specific to Western art and science – in the form of a Cartesian grid which appeared to the wearer of Sutherland’s head-mounted display.