2.3
Summary
Two major interaction styles have surfaced over recent years. Gesture-based and tangible in- teraction. Since in a typical tabletop setting no keyboards or mice are available performing complex commands can be challenging it seems promising to use gestures for command invoca- tion. Many interesting interaction techniques and applications based on this principle have been demonstrated. Gesture-based systems allow for an immediate interaction, the possibility to di- rectly touch information and spatial coupling of input and output make this interaction paradigm a promising candidate to the special design constraints imposed by the tabletop characteristics.
Tangible interaction promises to design richer interactions making use of manual dexterity and motor skills. The 3D nature of many physical objects affords specific ways of usage and as such helps to mitigate learnability and long-term memorization issues. Many researchers have speculated that motor memory would bear efficancy benefits over other interaction styles. Finally many positive aspects for embodied interaction and social interaction (e.g., access allocation, shareability) have been accredited to tangible interaction.
TUI and gesture-based interaction have often been studied in isolation even though they share many characteristics. To better understand the differences and commonalities of these two ap- proaches we discuss explorations into both interaction paradigms in the following Chapters. We then analyze their suitability, strengths and shortcomings for tabletop interaction. Based on this analysis we suggest a new model for tabletop interaction that allows for richer human computer interaction more akin to the flexible and continuous ways in which we manipulate objects in the real world.
In this vein we explore a new model for tabletop interaction in the following chapters of this dissertation. Combining some of the positive aspects of physicalityhighlighted in the study of our initial prototypes with a interaction style that is more open-ended andflexibleallowing users to make sense of the virtual world through (physical) exploration and experimentation.
Part II
33
“Take a method and try it. If it fails, admit it frankly, and try another. But by all means, try something.”
– Franklin D. Roosevelt
Drawing upon the analysis of related work in the field of digital tabletop systems presented in Chapter2this Part introduces our own initial explorations into tabletop interaction styles. Chap- ter2revealed two interaction paradigms that can be found in one or the other form across various proposed tabletop systems: gesture-based interaction and tangible1 interaction. Both interac- tion paradigms promise appealing characteristics worthwhile exploring in the particular context of tabletop computing such as potentially more intuitive manipulations and a better integration between the digital and the physical space.
Often systems applying one of these interaction paradigms have been evaluated in formal user studies but within the context of their application domain and with respect to the aspects of particular interest to the underlying motivations. In other words evaluations have often focused on whether the design choices were sufficient or appropriate to efficiently complete particular tasks or interact with a single system. In this Chapter we perform a meta analysis of two systems we have built and evaluated – our original evaluations were designed to answer similar questions, for example "do our design choices work in this particular context?" and are detailed in the respective publications.
In this Part we want to take a step back and use the chance to revisit the results and obser- vations we made during long hours of studying our prototypes. In order to be able to judge the potential of the interaction styles as generic paradigms for tabletop computing. Anticipating that interactions with digital tabletops will be different from standard computers, more ad-hoc, dy- namic and also more embedded into the social interactions with other users we useflexibilityand
physicalityas criteria for our analysis.Flexibilityrefers to the openness of the interaction; mean- ing how well does the interface lend itself for experimentation on the users side? How easy is it to discover functionality through simple experimentation and application of real world knowledge? How well can systems based on this interaction style be designed for appropriation? Physicality
refers to the level of realism in the behavior of interface elements and how well the interactions exploit our fine-grained motor skills and manual dexterity.
Our insights from these explorations serve then as input for a new model for tabletop interac- tion discussed in detail in PartIIIof this dissertation.
1in the context of tabletop systems the TUI approach is often combined with direct-touch interaction; we refer to
Chapter 3
Gesture based Interaction on
Tabletops
Our discussion in Chapter2highlighted a variety of tabletop technologies and applications that have been proposed in recent years. Clearly this new class of devices has caught the imagination of researchers and practitioners alike leading to a raft of experimentations with and applications for digital tabletops. Although as of now it is not clear what the model of interaction for tabletop computing is, early research in the field has shown that tabletop interfaces are subject to different design constraints than traditional GUIs. For example, one of the appealing characteristics of digital tabletops is their natural support for co-located collaboration – users can sit around the table and maintain eye-contact while they interact with the computer. When considering that each user has a unique viewpoint it becomes immediately clear that many WIMP concepts such as menus and buttons at fixed locations become a hurdle for simultaneous interaction with the system. Furthermore, orientation of on-screen objects plays an important role for text readabil- ity and for group collaboration, interaction and coordination [KCSG03, KCST05] (orientation
problem).
Another appealing quality of tabletop interaction is the possibility to directly touch informa- tion in a way often commented on as being natural. While it has been shown that direct-touch interaction can be effective in pointing tasks [SS91] (for bigger targets) direct-touch interaction can be problematic when interacting with typical (smaller) elements often found in traditional interfaces such as menus, buttons or sliders. This is mostly caused by two problems; the users finger occludes the region of interest in the critical moment before touching the display and the finger’s size is many times larger than individual pixels which makes it difficult to accurately select small targets such as buttons (occlusionandfat-fingerproblem [WFB+07]).
The absence of keyboards and mice from tabletop setups together with the orientation, occlu- sion and fat-finger problems have lead to experimentations with alternative interaction paradigms for digital tabletops. Gesture-based systems allow for an immediate interaction, the possibil- ity to directly touch information and spatial coupling of input and output make this interaction paradigm a promising candidate to the special design constraints imposed by the tabletop char- acteristics.
36 3. Gesture based Interaction on Tabletops
Of course “gesture-based interfaces” is a loosely defined term and can be applied to many dif- ferent approaches. In the context of this thesis we focus on finger or pen-trace gesture interfaces as discussed in Section2.2.2. Pen-trace gesture interfaces usually require some form of gesture recognizer that interprets geometric shapes drawn by the user with a finger or a stylus. These interfaces are often accredited with three main advantages:
Discoverability because gestures can be designed so that they resemble an activity or metaphor from the real world they are believed to be particular easy to learn.
Ease of use for similar reasons it is often claimed that gestures are easier to operate and use than interfaces based on abstract menus, buttons or textual commands.
Visibility of action given appropriate visual feedback gesture-based interfaces can provide users with awareness of their own actions due to tight spatial coupling of input and output and also provide group awareness in collaborative settings. In the latter case users’ interaction with the interface helps to reveal to one another what the task at hand is, and also helps collaborators to understand which parts of the information are presently being inspected (“look at this”).
In the following sections we discuss our exploration into gesture-based interfaces for inter- active surface computing. In order to gain a better understanding of this interaction paradigm we built and studied a prototypical application – calledBrainstorm– for collaborative problem solving in an environment enhanced with several interactive surfaces. The system uses gestures to invoke almost every single command in the system and we carefully designed the interface to address the orientation and occlusion problem. We designed several gestures ranging from literal real world metaphors to more abstract gesture to interact with a graphical user interface. the UI itself draws many clues and inspirations from the real world to increase discoverability and ease of use. We discuss our design choices and present results from a lab based user study. Finally we reflect on our observations from many hours of formal evaluation and informal system use.
3.1
Brainstorm: A Case Study
Brainstormis a system built using several interactive surfaces including a digital tabletop and several wall-mounted displays. The system is intended for supporting co-located collaborative problem solving. Collaborative problem solving requires knowledge and information to be ex- changed among team members; different skills have to be coordinated and the information com- municated by others needs interpretation, so that new ideas can be created and new solutions can be found. This process - with its core requirements of communication, coordination, and interpretation - is called collaborative creative problem solving [Ama96].
The design of the Brainstorm socio-technical system is meant to explore the possibility of merging the physical and social qualities of a traditional face-to-face collaborative creative environment together with some of the benefits of digital technology, such as persistent data