2.4.3 Audiophotography
Frohlich [2004] introduces the idea of audiophotography, defined simply as photographs with associated sound. This audio might be ambient sound when the photograph is taken, a piece of music, the subject of the photograph talking, or conversation about the subject – “photo-talk”. The latter forms a type of audiophotograph called conversational photographs. In studies using traditional, printed photographs, he identified two types of photo-talk:
reminiscing, which was characterised by equal participation in conversation, overlapping talk
and usually occurred when all participants shared the memory depicted; and storytelling, usually occurring when a participant was not involved in the memory, characterised by little overlap, unequal participation and a tendency for the photographer to focus on particular photographs in a collection. Frohlich presents evidence for the value of capturing and preserving conversation around photographs [Frohlich et al., 2002], but also states that the reminiscing style is unlikely to have long-term value as a voice-over [Frohlich, 2004] due to the more unstructured conversation style. Preserving storytelling, however, is suggested to have benefit.
As yet, no work on digital photograph sharing has attempted to automatically capture the photo talk.
2.5
Chapter Summary
Tabletop interface design poses some challenging constraints. Being able to sit opposite the storyteller (as in Figure 1.1) permits eye contact to be made, indicating awareness and attentiveness in the story, and direct manipulation allows for gesturing and pointing. However, without a keyboard or mouse, there is a sparse set of interaction operations, typically just touch, dwell and drag (and in multi-touch some combinations). Furthermore, we must overcome problems stemming from the pragmatics of tabletop interaction such as orientation (text and otherwise), large displays (resolution, user reach), and coordinating interaction. These are important differences compared with conventional computers.
To assist application designers in supporting the tabletop interface, an effective frame- work is required that considers the constraints of tabletop interaction, and provides flexible and new ways for users to interact with the tabletop, as well as with each other. One cannot assume that interface elements designed for a single user on a vertical display designed to be used indirectly, via a keyboard and mouse, will have the same effectiveness when used on a multi-user direct-interaction tabletop interface. Toolkits for tabletop interaction have not yet gone far enough towards providing the necessary application support for the rapid development of novel, flexible and functional tabletop applications.
The tabletop interface and an effective application framework give us a special oppor- tunity to leverage social aspects that the interface can provide in order to share our digital photographs. Such an environment has real potential to bring back some of the emotional attachment that is accorded when interacting directly with printed photographs, as well as providing a digital environment to facilitate sharing our collections of digital photographs, without having to print them out. By virtue of being digital, we also have the opportunity to automatically capture this interaction to implicitly create an audio-augmented digital photo album of our stories as a side-effect of the photo sharing. Realising this goal, whilst providing an extensible application framework with novel and effective interface elements for browsing through large collections of real users’ digital photographs is the focus of this thesis.
Chapter
3
Design Overview
We begin with a high-level design overview, and discussion of our design rationale, before going into depth about how the design is realised in Cruiser and PhoTable. There are two aspects to design in this thesis: design of the software framework, and design of the user interface. Throughout development, these aspects have influenced each other – the Cruiser framework from the bottom-up and the PhoTable user interface (combined with the photo sharing motivation) from top-down.
Because the interface depends on the framework (and the framework is intended to be able to support a variety of interfaces), it is tempting to begin by describing the framework. However, programming necessitates abstractions, and the Cruiser framework has many abstractions to deal with the complexities of software development. To provide the design influences that establish the requirements for these abstractions, this chapter will begin by describing design of the user interface.
This chapter, too, is necessarily abstract, as we give rationale for our design before presenting actual design, which is then followed by the implementation behind the design. If you are unfamiliar with the Cruiser framework, you may wish to refer to figures in Chapter 4, or accompanying video you may have received with this thesis, to give the rationale some initial context.
3.1
User Interface Design Drivers
Before describing the specifics of design for the user view of the PhoTable application, a more general description of design for tabletop interfaces is needed. This chapter first describes the particular problems that must be overcome in design for tabletops, followed by the key elements for interaction that were identified for use in Cruiser.
Given the novelty of tabletop interfaces, there has been little work with a focus on design for tabletop interfaces. Some important foundation work includes Scott’s [2005] studies of users managing photographs at a tabletop as a basis for the design of personal spaces and storage bins. From many general guidelines [Nielsen, 1993] of user interface design, and an early influence of considering usability for the elderly [Apted et al., 2006], we established principles to guide the design of our tabletop interface:
G1. Focus on learnability and memorability.
(a) Rely on familiar aspects of manipulating physical documents (e.g. photographs) on a tabletop. This reduces the amount to learn and remembering is easier since the user already knows how to move and share physical photographs.
3.1. UI Design Drivers CHAPTER 3. DESIGN OVERVIEW
(b) Minimise the number of interface elements, based on common user interface goal of simplicity.
(c) Strive for predictability by maximising consistency.
(d) Use new objects with new appearances for new interface behaviours [Tognazzini, 1996]. This avoids clashes with the user’s existing knowledge.
(e) Provide continuous and appropriate feedback (usually contextual visual feedback for the tabletop, as audio can be problematic with multiple users).
(f) Avoid modal operation where possible, as this requires switching and memory of the current mode; particularly avoid irreversible changes.
G2. Address special aspects associated with multi-user tabletop user interfaces: (a) They may be large so a user may be unable to reach the whole table easily. (b) Ensure the user can easily make interface elements larger. This ameliorates the
effects of poor display granularity and also, by allowing the user to enlarge user interface elements as much as they please, they can reduce the need for fine motor co-ordination and reduce the effects of the “fat finger syndrome”. (c) Ensure interface elements can be rotated, for viewing by different people, situated
at different sides of the table, or at arbitrary angles.
(d) Support the user in reducing clutter. Clutter is a particular issue for tabletop interfaces. In addition, this is especially important if many user interface elements need to be large. In general, we should reduce distraction from the current focus.
(e) They are social spaces, commonly used with small groups of people. G2bis also significant here; e.g. when users enlarge an image in order to discuss it. (f) They form part of a pervasive computing environment (e.g. lacking keyboard
and mouse), and may need to interact with other parts of the environment (e.g. throw a photograph onto the wall, attach a digital camera, etc.).
G1 is largely adapted from classic guidelines such as Nielsen [1993], which emphasise predictability, consistency, feedback, error prevention and reduction of memory load. The tabletop gives additional focus. For example, Wu et al. [2006] noted the importance of providing continuous feedback (G1e) at a gestural, tabletop interface. In addition, in the absence of an undo operation, feedback is particularly important to help prevent errors. Providing an undo operation remains a problem exacerbated by having multiple users and simultaneous interaction.
Similarly, a modal interface (G1f) often requires all users to be aware of, and agree upon the current mode, so is best avoided. A common use of modes in traditional graphical interfaces, such as drawing programs, is the toolbar. Selecting an item from the toolbar puts the interface into a new mode, such as draw or select, and is usually indicated by changing the mouse cursor, which gives an iconic indication of the active mode at the same location as the user’s focus. However, even if users can be reliably distinguished, the absence of a cursor in direct-interaction tabletop interfaces means that the indication of the current mode must always be away from a user’s current focus.
G2 addresses problems that are special to tabletop interfaces, and is given some focus by the context of our photo sharing application. Clearly, the set of guidelines cannot be blindly applied to all tabletop applications. Some applications, particularly more basic or constrained tabletop applications (e.g. gaming), will not be affected by all of the same issues. However, the full set of guidelines are important for a generic framework.
G2bwas partially influenced by our initial focus on elderly but has wide applicability – a user cannot pick a photograph up to examine it closely and the large dot pitch means that leaning closer to the table (e.g. as observed by Terrenghi et al. [2007]) helps only slightly. G2cand G2earise because of the multi-user nature of tabletops – if we are to leverage the benefits of multi-user and social interaction that interactive tabletops can provide, we must design software aspects of the interface to support this.
The issue of clutter (G2d) arises because of the fundamental differences from physical tabletop interaction. For example, it is easy to create new images and multiple people may interact in parallel (e.g. with copies of the same photograph). The issue of clutter is a prominent issue amongst research involving interactive tabletops and the kinds of tasks they are designed to support (e.g. as observed by Shen et al. [2002], Morris et al. [2006b], Ryall et al. [2006], Hartman et al. [2006], Hilliges et al. [2007], Leithinger and Haller [2007]).
We now discuss how these guidelines, and the restricted input available (G2f) in the tabletop interaction paradigm, influence the high-level design of user interaction.