Preliminary considerations on interaction styles

3.4 Aspects related to the interaction

3.4.1. Preliminary considerations on interaction styles

Examining the wide corpus of installations from the point of view of the devices that are used⁴¹, a first distinction emerges between those that make visible use of traditional devices (like screens, mobile devices, touchscreens) and those that use devices that take on the appearance of low-tech objects (smart objects). In addition, it can be noticed that, in some cases, in order to interact with the system a contact with the device/object (through touch, manipulation) is required, while in other cases the contact is not necessary and the visitor can interact just making gestures in the presence or in front of the device/object. By intersecting these two dimensions (with/without contact and traditional devices/smart objects), a graphical representation can be obtained allowing for the identification of different categories of interfaces and interaction styles (Figure 55).

Figure 55 Different categories of interaction styles emerging from the intersection of two dimensions (contact/without contact; traditional devices/smart objects).

41 At this stage of the analysis, a distinction is not made between input and output devices, and both are considered. This is because it has been noticed that in this kind of projects the distinction between input and output devices is often misleading and generate ambiguity.

A first category of interfaces/interaction styles (top-left) is represented by traditional interfaces/interaction. They usually require an interaction through contact with just traditional devices (mouse, keyboard, touch-screen, mobile devices) and an output is provided through a traditional output device as well (e.g. screen, loudspeaker, headphone). This traditional type of interaction is not the focus of this thesis and has generally been excluded during the collection of existing projects.

At the top-right and bottom-right there are two other categories of interfaces sharing the fact that they view the presence of only low-tech objects as input and output devices. However, while the latter category requires a physical contact with an object (e.g. Figure 56), in the former a contact with the object is not needed as the systems included in this category view the presence of objects that are able to recognize free gestures the visitor performs – more or less consciously – in front or nearby them (Figure 57). Sometimes being referred to as object-based interaction the former, and gesture-based interaction the latter, in this thesis the expressions “manipulation-based tangible interaction” and “free-gesture-based tangible interaction” are preferred as the presence of objects or gestures are not discriminants between the two types of interaction, being them present in both cases. Categories two and three are ascribable to a design attitude that can be summarized with the term

“disappearing computer” characterizing a good portion of tangible interaction efforts. It is important to notice how these projects in reality do make use of traditional devices (like screens, mobile phones, loudspeakers etc..) as well, but these are made invisible by hiding or embedding them inside other objects or by integrating them within a context that gives them a different appearance.

The fourth category concerns those projects that make use of traditional devices (e.g. a screen) but, at the same time, they require the visitor to interact through free gestures going beyond the traditional interaction styles (through the mouse, keyboard, touchscreen etc.) (Figure 58).

These types of projects are often referred to as natural user interfaces in literature. Although a wider interpretation of tangible interaction may see these kinds of interfaces as belonging to the tangible interaction paradigm, we prefer considering them as something slightly different.

This is one of the reasons why not so many projects of this kind have been included in our corpus for analysis.

Figure 56 In the talking tactile panel at the San Diego Museum of Art, exploring the surface of a replica of the painting the visitor can hear a voice explaining the meaning of what he is touching (source: Touch Graphics, 2015a) [App., Proj. 1].

Figure 57 The first part of the Magic Mirror installation, consists of a typewriter placed on a pedestal. In the idle state the visitor can hear sounds of tapping, and the object is spotlit. As the viewer goes closer to the object, the intensity of the light varies according to his proximity, and the sounds of tapping stop (source: Rawat, 2005) [App., Proj. 47a].

Figure 58 In “Strike the pose” the visitor interacts using her body in front of a screen trying to match the pose of sculptures (source:

https://vimeo.com/60866008) [App., Proj. 52].

However, the reality of tangible interaction is more complex, and a rigid classification would not help to reflect about its complexity. So, on the border between the different categories illustrated above other styles of interaction emerge that, anyway, can be ascribed to the category of tangible interaction. In particular, smart objects and traditional devices often coexist in a same project albeit with different functions. The smart object usually acts as a controller to interact with a traditional device (e.g. a screen) that, in turn, acts as an output device (Figure 59). In addition, on the border between manipulation-based tangible interaction and free-gestures based tangible interaction, there are projects where both types of interaction are supported and/or needed (Figure 60).

Figure 59 In VIRTEX (Augustus of Prima Porta) installation, pushing specific sensors on the physical scale model of the statue, a virtual view of the specific area along with related information are provided on a screen

(source: https://www.youtube.com/watch?v=nQbdn2NVlls)[App., Proj. 7].

Figure 60 “Wonder objects” combines both free-gestures and manipulation-based interaction. In the idle state, the visitor sees and hears hands typing on the machine’s keyboard. As the visitor goes closer another video is triggered showing a set of instructions. Then, pressing specific hotkeys, video content is projected on the typewriter’s sheet of paper animating the object (source: Rawat, 2005) [App., Proj. 47d].

This research is mainly focused on projects that either contain smart objects only or a combination of smart objects and traditional devices, and only marginally on those projects that are made just of traditional devices although requiring a more natural interaction.