Designing Frameworks

Designing interactions for TUIs leverage several challenges that could not be addressed by previous research on GUIs. Dourish [60], introducing the idea of embodied interaction, has been an important source of inspiration for the tangible interaction community. The concept of embodiment stems not only from the idea of embodying digital information in tangible artifacts, typical of physical computing [150] and early TUIs, but also from a phenomenological approach to the creation of meaning. Indeed, instead of thinking to the knowledge and to the meaning of interactions as existing per se, the phenomenological approach, following the theories of embodied cognition [216], suggests that the meaning is created by the humans while exploring its surrounding through their senses. From this point of view, designing embodying interactions means that the designers should provide the user with affordances that can be discovered while acting with artifacts and engaging with exchanges with other users, which involves sociological questions into the design of the interactive systems. He also advocated the necessity of a paradigm switch, with interactions that are ready-to-hand (following Heidegger philosopher’s terminology), where the user can benefit of the system without stopping to theorize about it and to understand its meaning. In this latter case, instead, which is typical in HCI according to Dourish [60], the system becomes present-at-hand. Although the ready-to-hand paradigm has been criticized recently, suggesting the necessity at some point to step back from the interaction and to reflect and possibly learn new skills or knowledge [92], Dourish theory and his six principles⁹ for designing embodied interaction have profoundly inspired the following trends in tangible interaction.

Several researchers provided more practical guidelines for designing tangible interaction.

Bellotti et al. [25] identified five questions that the designer of TUIs (or more in general of interactions for Ubiquitous Computing) should answer: Address, the designer should ensure that the users can communicate easily with the system; Attention, the system should inform the users that it is ready to listen their commands; Action, the designer should establish which actions the users could perform with the system; Alignment, the system should acknowledge the users that it understood the users’

requests; Accident, the users should be able to recover from an error and undo unintended commands.

9 Note: these principles are mostly related to the aforementioned concepts and will not be detailed in this thesis for brevity purposes.

Sharlin et al. [188] evidenced three heuristics for exploiting spatiality in TUIs: physical/digital mappings should be translated in congruent spatial mappings; input and output space should be unified;

the physical/digital mappings should allow the user to try different configurations, even wrong, until the right solution is found. Djajadiningrat et al. [58] propose a different perspective for the design of the interaction of tangible products: the designer should focus on direct interactions that can be performed exploiting the affordances provided by the product, rather than on symbolic interactions that rely on the previous knowledge of signs and metaphors. They also advocated a switch from a focus on visual appearance to products that are more appealing for the provided interaction. Their final suggestion is to design products that can exploits all the human skills, from cognitive to perceptual-motor skills and emotional skills. Still from a product design perspective, Wensveen et al. [224]

proposed a framework for coupling action and function in tangible products through feedback and feedforward. They identified six aspects that should be coupled between user action and product reaction: time, location, direction, dynamics, modality and expression. Feedback (the information provided to the user during or after the user’s action) and feedforward (the information provided to the user before the user’s action) can be designed on three different levels: the functional, i.e., the function and services offered by the product, augmented, i.e., complementary information that exploits cognitive skills to indirectly inform about the product functions and inherent, i.e., information coming directly from the performed action through the user’s perceptual-motor skills.

Among abstracting and designing frameworks presented until 2006 it is possible to notice already several different perspectives about tangible interaction. While the first frameworks and models focused on a data-centered vision and on applications for professional use, newer frameworks focused more on the user engagement started appearing. In 2006, Hornecker and Buur [95] formalized this new trend by identifying four main themes and several concepts that designers can investigate for each theme (Figure 9). The first theme is tangible manipulation and investigates the typical concepts of data centered systems where data can be directly manipulated through continuous interactions. With spatial interaction, they discussed not only the typical concepts of spatiality of Sharlin et al.[188], but also the possibility to exploit the surrounding space or full-body interactions, including performative and communicative body movements. With embodied facilitation, they stressed the importance of physical space and artifacts to leverage user behaviors such as collaboration, social exchanges and exploring physical skills that go beyond intuitive interaction. Finally, with expressive representation, they highlighted the importance of having significant physical representation of the digital data, with a perceived coupling between user’s actions and system reactions and long-lasting relationships between artifacts and data that could help users’ discussions. Hornecker physically implemented these suggestions for designers with a set of cards that contain questions and images concerning the different

concepts and themes [93]. The card set should be used to stimulate the designer reflection during the conception of a new tangible interactive system.

A milestone framework for post-WIMP interfaces, including Tangible User Interfaces, is Reality-Based Interaction [99]. Jacob et al. [99] discussed the importance of leveraging different human skills in the interaction with real-world ubiquitous interfaces: naïve physics, the common understanding of the physical world; body awareness and skills, the knowledge of our body and of its potentialities; environment awareness and skills, the knowledge about the surrounding environment and the ability to interact with it; social awareness and skills, the knowledge about other people and the ability to communicate with them (Figure 10). Engaging these human skills is crucial in reality-based interaction, but they should be traded off with more practical considerations of the application, i.e., the expressive power, the efficiency, the versatility, the ergonomics, the accessibility and the practicality.

Figure 10. Reality Based Interaction Framework [99]

As already discussed at the beginning of Section 2.2, the broadening of tangible interaction themes and concepts makes a question arise: “What is then tangible interaction?”. Hoven et al. [214]

tried to answer this question, without giving a sharp definition of tangible interaction, but individuating the common foundation of tangible interactive systems and the main qualities that can be found in these systems (Figure 11). The foundations are basic elements that are highlighted also on previous frameworks and models: the presence of a physical world, the exploitation of human skills and an underlying digital computation. The qualities of tangible interaction individuated by Hoven et al. are direct representation and control, i.e., the possibility to manipulate and get aware at glance of digital information; integrated representation and control, i.e., the action and perception spaces are merged in the physical world and integrated in the tangible artifacts; and meaningful representation and control, i.e., the interaction is easy to understand for the user and it is conveyed through physical affordances, metaphors or image schemas.

Figure 11. Foundations and qualities of Tangible Interaction [214].