Devices: an analysis on the form and role of smart objects

Installations like these can be considered to implement the model for heritage enhancement envisaged in the context of the European Project MeLa* (European Museums in an age of migrations), the “Performing Heritage Model” (Lupo et al., 2014). According to this model,

“intangible heritage is experienced in the first person by the user, who can play and perform cultural practices, alone or in a shared situation”

(Ivi), that is, “cultural contents is performed and re-produced, individually or in a social context, in a space that becomes a stage, created by users’ gestures” (Ivi).

The use of performing gestures can be seen as a strategy that allows to associate intangible values and meanings to tangible objects in tangible interaction installations, by integrating them implicitly into gestures.

This strategy can be referred to as “embodying meaning” (Duranti, Spallazzo and Trocchianesi, 2016a), because by performing specific gestures or bodily interactions, the visitor implicitly understands and experiences an intangible value related to a certain object.

It is important to point out that the already presented distinction between interaction with contact and without contact is still valid.

Codified actions can either require the contact with the object (e.g. Ara Pacis) or not (e.g. Constable) and the same can be said of performative actions⁴⁶.

3.4.3. Devices: an analysis on the form and role of smart objects

In order to achieve specific objectives or accomplish specific tasks, the visitor has to perform certain actions (with or without contact) at the level of the interface and get the desired feedback from the interface itself. The interface implies the presence of one or more devices that, as said, can be categorized as traditional devices or smart objects depending on whether their digital nature is explicitly declared (e.g. a smartphone) or hidden inside the appearance of non-digital objects (e.g. the meSch Loupe [App, Proj. 27]). The devices can act as input devices, as output devices or, sometimes, as input and output devices at the same time.

46 Although all the performative actions presented so far required the contact with an object, performing actions without contact with an object can be envisaged. As an example, it would be possible to implement the Virtual Conductor installation, in such a way that the visitor does not need to handle the baton, but just simulates the movements he would do with his hand, although the effect would likely be less realistic.

This paragraph focuses on smart objects, being them a distinguishing feature of tangible interaction, and proposes an analysis and categorization of smart objects from the point of view of their form and relation with the main cultural heritage asset.

A first category of smart objects concerns those that present a direct formal relation with the tangible cultural asset the interactive installation refers to (primary object). These include smart objects consisting of:

• smart original: the very original object that is made smart (Figure 62);

• smart replica: more or less accurate copies of the reference object (Figure 63);

• smart derivative: objects that are derived from the reference object through processes of abstraction or translations (Figure 64; Figure 65).

From a practical point of view, the use of original objects has the advantage of allowing to overcome the problem regarding the distraction from the original object that characterizes traditional uses of technologies in museums, since the original object is incorporated in the installation itself.

Instead, smart replicas provide an alternative to the use of original objects in tangible interaction stations, e.g. when the incorporation of the latter is not possible for preservation reasons. However, it must be noticed that, with the use of smart replicas the problem of distraction could even be amplified in that they could present themselves as identical copies or even more appealing substitutes of the original objects, unless proper design strategies are adopted to limit this negative effect. However, smart replicas offer some advantages in terms of flexibility compared to smart originals. For example, it is possible to print small objects in larger scale in order to make details more visible, to make the object easier to grasp or able to accommodate the desired sensors (e.g. Virtex – Ivory Object).

Figure 62 In the “Olivetti MP1” exhibit, the visitor interacts directly with the original object. Pressing specific hotkeys on the typewriter, video contents are projected on the typewriter’s sheet of paper animating the object (source: Rawat, 2005) [App., Proj. 47d].

Figure 63 The Interactive stela installation consists of a plastic sensorized replica of an Egyptian stela that is present in the museum (source: the author) [App., Proj. 45].

Figure 64 In “Interactive histories” the interactive objects are physical tokens consisting of abstract representation of objects that are in the collection (source: Kettner, 2013) [App., Proj. 42].

Figure 65 In the “Talking Painting” installation a Juan Sànchez Cotàn’s still-life painting is “translated” into a bas-relief that can be accessed by blind people (source: Touch Graphics, 2015a) [App., Proj. 1].

Another category of smart objects concerns those that take on the appearance of an object that is different from the main heritage asset the installation aims to enhance (secondary object). Although different from the main cultural heritage asset, sometimes this object presents a cultural relation with it in reality (related object). An object can relate to the main object for example because it belongs to the same historical context of the main object (Figure 66; Figure 67), or to one of the contexts of activities that characterizes or characterized the main object⁴⁷. An example is the Loupe (Figure 68), inspired to an existing tool that is actually used in real life both by expert and ordinary people for the exploration of a detail of an object, such as a vase. Similarly, related objects are also all those smart objects that are inspired to tools that are traditionally used in the context of the visit of a cultural heritage object/site, such as traditional visiting aids (leaflets, books) or exhibition equipment (glass cases, drawers, etc.) (Figure 69; Figure 71).

Finally, there are some projects in which the smart object relates to the main object because it represents the contemporary counterpart of an historical object (Figure 70).

Figure 66 In the “Companion Novel – Voice from the Trenches installation”, the main reference object is the open-air “Trenches of Nagià Grom” site. The smart object consists of a wearable belt inspired to the WW1 army clothing activating audio contents during the exploration of the site (source: Marshall et al., 2015) [App., Proj. 22].

47 The notion of “context of activity” comes from activity theory. For more information about the notion of “context of activity” and activity theory and its application to the interaction design field see Kaptelinin and Nardi (2009), while see Kaptelinin (2011) for more information about how activity theory can be applied to the field of ICT in museums.

Figure 67 In the “Atlantik wall installation” the main objects consist of documents and artefact related to the life and events at the time of the Atlantik Wall in the Hague.

In order to trigger stories related to the exhibited artefact, secondary related objects are used as representative of different perspectives (civilians, Germans, officials) (sourc e: https://www.youtube.com/watch?v=sK3AdQU9kkc) [App., Proj. 12].

Figure 68 The “meSch Loupe” for example is a sort of magnifying glass, which beyond providing information overlapped over the object, also provides a magnifying function (source: Petrelli, 2014b) [App., Proj. 27].

Figure 69 The Companion Novel installation made for the Sheffield General Cemetery consists of an interactive book that visitors can carry with them during the exploration of the cemetery (source: Ciolfi et al., 2013b) [App., Proj. 21].

Figure 70 In the “The Really Simple Object Recognition Interactive Anglo-Saxon Table” made for the Maidstone Museum and the Bentlif Art Gallery, contemporary objects are used to activate contents about Anglo-Saxon counterparts. In the picture, a contemporary plastic mug is used to get information about an Anglo-Saxon Glass Cup (source: https://www.youtube.com/watch?v=t UBVwJzFHn8) [App., Proj. 55].

Figure 71 In the Fryderyk Chopin Museum an exhibition equipment is made smart. Opening the drawers, the visitor can observe Chopin’s original scores and at the same time listen to the music and see projected information on the table

(source: http://www.designboom.com/design/chopin-museum/) [App., Proj. 40].

In other cases, smart objects have the appearance of objects that have no relation in reality with the main heritage asset the installation refers to (Figure 72; Figure 73). We can refer to them as unrelated objects. Often these objects are abstract objects like cubes, balls, etc. that are used to activate contents about the main object. It is the responsibility of the designer to define the type of relationship the unrelated object has to establish with the main object once it is inserted in the installation.

Beyond being an activator of content, in many cases the object becomes a symbolic representation of aspects or intangible values that characterize the main object (e.g. different interpretation or points of view).

Figure 72 In “ec(h)o”, a wooden cube having three coloured sides is used by the visitor to select specific audio contents in front of specific natural objects exhibits (source: Wakkary et al., 2007) [App., Proj.

32].

Figure 73 At the Museo delle Culture di Lugano, using a ball the visitor can select a visit profile according to which information is provided on a monitor (source: Bacconi, 2011, August 26) [App., Proj. 43].

It is interesting to highlight that often the boundary between smart objects and traditional devices is very subtle, as in many cases traditional devices like screens, loudspeakers, and mobile phones are actually used as part of smart objects. However, what makes traditional devices different from smart objects is the fact that, while in the former the digital devices manifest themselves as such, in the latter digital devices are hidden or integrated inside objects, and this gives them another aspect and interpretation. A smartphone inserted in a loupe is no longer perceived as a smartphone, as well as a screen showing a painting and that is inserted in a physical frame is no much perceived as a screen but as a painting.

Smart objects and Intangible values

This paragraph provides some reflections regarding the relation between smart objects and intangible values, aiming at shedding light on different ways of using smart objects for associating intangible values to cultural heritage. This analysis is carried out with reference to the two broad categories of smart objects that have been defined above, namely primary objects and secondary objects.

In most of the projects that make use of primary objects, these either act as physical activators and/or output devices for intangible values that

are related to the object itself. By interacting with the object, the intangible values connected to it manifest themselves explicitly in the form of digital content. This use of smart objects allows for a better integration between the material aspects of the object and its intangible value, as well as for a reinforcement of the link between these two elements, that are often presented in more separated ways in museums.

A second category of smart objects concerns secondary objects.

Belonging to this category there are all the smart objects that take on the appearance of objects that are either related for some reasons to the ones the installation aims to enhance, or that seem to be completely design actions that can be used to convey the intangible values related to the objects through the use of smart objects, a practice that can be referred to as “embedding meaning” (Duranti, Spallazzo and Trocchianesi, 2016a).

In the projects that make use of primary objects, the intangible values related to an object are communicated by means of digital content that can be activated by physically interacting with the objects itself or its replica. In this case, the design action concentrates mainly on the definition of the content and how to link it to the object, and less on the object itself, that has mainly a denotative function⁴⁹, as it represents itself. In the projects where secondary objects are used, the objects are not always just activators of contents, but can sometimes become symbolic of certain intangible assets. In this case, the design action goes beyond the definition of contents, to include an effort towards the definition of the shape of objects that is suitable to become symbolic of certain intangible values (connotative function).

48 Sometimes, however, more than symbolic of an intangible value related to the main object, the smart object is symbolic of a functionality of the system, as in traditional TUIs.

49 The concepts of connotation and denotation have been borrowed by semiotics.

“Connotative meaning is a secondary meaning which a sign may have in addition to a primary, standard, or core meaning, called denotative” (Nöth, 1998, p. 102).