For the user interface design, we intend actions to be as natural as possible through the use of a variety of visual affordances. Some of these affordances are derived from equivalent, purely physical interactions that occur with printed photographs placed on the tabletop. We do this to reduce the semantic gap and to minimise the number and complexity of new concepts our users were required to learn (G1a). To maintain the link with the physical world, users interact only with photographs – there are no buttons, menus or toolbars to be navigated, nor any special keyboard or gestural commands (G1b).
Special photographs, with a different appearance (G1d), but consistent behaviour (G1c)
perform special functions. These special photos are moved, rotated and resized just like regular photos, but trigger their special function when dwelled upon, or when another object interacts with it. The interaction of objects in this manner gives additional opportunities to provide continuous feedback (G1e) and avoids modes (G1f) because, rather than having a toolbar, the “tool” itself is available on the interface. Feedback is given throughout the special actions, so the user can immediately see the effects of their interaction. The actions are also designed to support an early exit with minimal, reversible changes if the user decides, from the feedback, that it was not their intended action.
This section now discusses the user interface “keystrokes” that are available to designers of tabletop interfaces. This provides the basis for a form of evaluation when exploring potential alternative designs for an interface action.
3.2.1 Direct-Touch “Keystrokes”
The lack of a keyboard or mouse (G2f) means that we must reinterpret the meaning of
keystroke – there are no keys. In aGoals, Operators, Methods, and Selection rules (GOMS)
analysis, the original Keystroke-Level Model (KLM) [Card et al., 1980] represented the operators required to perform a user interface task in terms of Keypress (on a computer keyboard or mouse button), Point (with a mouse), Home Hands (to move hands between mouse and keyboard), Mental preparation, and system Response time. These were combined with heuristics for the placement of M operators, and have been updated for WIMP-style direct manipulation interfaces [John and Kieras, 1996]. However, these operators are not appropriate for direct-touch interfaces, such as tabletops, where there are no keys and no cursor to move.
3.2. Influences for User Interface Design CHAPTER 3. DESIGN OVERVIEW
This thesis does not attempt to formulate a new GOMS for tabletops. Indeed, for many of the tasks tabletops are well suited to (e.g. multi-user collaboration and, in this case, photo sharing) measuring task execution time is less important than effective support for the creative, and often artistic, processes involved around tabletop collaboration. In addition, GOMS is designed for expert performance, and we explicitly want emphasise
learnability as well as expert usability. However, it can still be useful to consider our
interface design in terms of such operators, or interaction primitives. For direct-touch there are surprisingly few:
Touch (or tap) where a user makes contact with the interactive display with a finger or stylus,
Dwell where a user maintains stationary contact with a point on the display, and Drag where the user slides their point of contact over the display.
A stylus with buttons can incorporate an additional component, but otherwise more complex interactions can be formulated in terms of these three. Even multi-touch can be represented simply by using multiple of these operators.
Shape recognition in tabletops may provide more operators through the detection of various hand or object shapes, but not all tabletop hardware has support for this and the robustness of such recognition can have a negative impact on usability. As the approach to development of tabletop software in this thesis is to be hardware-independent, shape recognition is left for future work.
Note that throughout this thesis, a “touch” or “touch point” refers to contact made with either a user’s own fingertip or with a pen/stylus. We do this mainly to simplify descriptions of the interaction techniques.
3.2.2 Design Approach
Since tabletop interaction is a new area for user interface design, we utilised a form of ex- ploratory design based on iterative refinement within our flexible development environment. The essential elements of the approach are:
• a development platform that is sufficiently flexible that we can explore diverse approaches, reducing the constraints on our thinking;
• brainstorm about the primitive actions that should be explored; • choose one and then brainstorm about the gestures to support it;
• conduct desk evaluations based on heuristic evaluation and keystroke analysis; • use this to refine the design;
• evaluate with user trials; and
• iteratively refine the design and repeat evaluations.
Here evaluations include informal user trials such as exhibitions and interactive demon- strations, and larger, more systematic evaluations of the interface (e.g. Apted et al. [2006], Apted and Kay [2006a], Collins et al. [2007], Apted and Kay [2008], Collins [2006]).
This is a fairly standard exploratory design approach with a short iteration cycle between refinement of the design. The challenging aspects for tabletop interfaces are to establish a suitable development platform, escape the straight-jackets of preconceptions from both experience in the physical world and conventional interfaces and identify suitable heuristics for the desk evaluation.
Function Why Core Select Establish Focus, raise to top, ensuring visibility
Move Pass photographs, organise, layout, etc. Rotate Orientation for user view and sharing
* Resize Make focus for discussion or view (enlarge), make smaller when not focus Copy Duplicate whole photos and control objects
Grouping Attachment, Storage Bins, Personal spaces – designate areas of the table, object associations, flip
Delete (Hide)
via the “Black Hole”: Reduce table clutter, remove mistakes
Capture via the “Frame”: crop photos and create new photos, allows further levels of zoom and a facility to retrieve finer detail from the original photograph, on disk
* Rotate and resize is a combined action –rosize
Table 3.1: Summary of Core Functionality in a Cruiser user interface