Touchscreen and Tangible Interfaces for Collaborative Music

Just as touch screens have become a major focus for Shareable Interfaces in CSCW research (see Section 2.2.5), touch surfaces for musical interaction, performance and composition are a rapid field of expansion (Xamb´o et al., 2011b), with touchscreen mobile telephones and tablet com- puters becoming commonplace tools for musicians (Oh et al., 2010). At a larger physical scale, interactive table interfaces such as the reacTable (Jord`a et al., 2005) have captured a great deal of public attention. The reacTable allows musicians to collaboratively patch together sound gen- erators and processors by manipulating and arranging small physical objects on a rear-projected tabletop interface. These physical objects represent different sound generators and processors; with position, rotation, and proximity to one-another mapped to various synthesis parameters. This physical interaction metaphor is intended to provide both an intuitive experience for musi-

3

cians and a compelling spectacle for an audience by providing a visual connection between the performer’s physical gestures and the music being created. Similar tangible object based inter- faces for music-making and musical interaction include BlockJam (Newton-Dunn et al., 2003) and Audiopad (Patten et al., 2002).

Purely touch-based music environments have also been developed, including those presented by Davidson and Han (2005), Fencott (2008), Iwai (1999), Xamb´o et al. (2011). Davidson and Han (2005) stressed that purely touch based interaction provides more flexibility than interfaces based on tangible objects, as touch-only allows for dynamic re-contextualisation of the graphical interface. Furthermore the interface components are not limited to the physical size constraints of the physical objects, and the screen does not need to be horizontal. However while noting the potential for the multiple-user interaction, Davidson and Han (2005) focuses on technological concerns, rather than social or collaborative aspects of multi-touch technology.

Regardless of whether these systems use direct touch or tangible object based interaction, a key feature of interactive surface interfaces is the support for multiple points of interaction by one or more simultaneous users (Davidson and Han, 2005, Kl¨ugel et al., 2011). This calls not only for consideration of musical interaction metaphors, but also a sensitivity to the nature of human collaboration and in particular issues of awareness, territory, sharing and privacy, as discussed in Section 2.1. Acknowledging this fact, Kl¨ugel et al. (2011) drew extensively on stud- ies of collaboration to inform the design of a collaborative multi-touch music interface. Using CSCW literature, the paper identified a number of constraints and requirements for collaborative interaction; group awareness, group articulation and tailorability. A key contribution this paper makes is to propose that non-spatial interaction metaphors which operate independently of the orientation or position of interface elements are preferable in a collaborative context to mapping strategies which rely on the spatial position of objects, as the non-spatial designs can more ef- fectively support the dynamic coupling of users. They also acknowledge the role of awareness in collaboration and suggest that interfaces should support tailorability by allowing adoption and use in personal and unintended ways.

Aside from the interactive surface interfaces based on conventional paradigms of computer music software (on-screen oscillators, musical keyboards, sliders and so on), multi-touch inter- faces are ideal for placement in public contexts where accessibility and immediacy are central concerns (Benko et al., 2006). Examples of interactive surfaces designed for playful engagement

or public exhibition include the interactive cellular automata presented by Fencott (2008) the Scambi Surface Sequencer (Fencott and Dack, 2011), and Composition on the Table (Iwai, 1999), all of which leverage the potential of interactive surfaces to support direct intuitive engagement with sonic and visual materials in a manner which is distinct from conventional music-making techniques and tools.

Xamb´o et al. (2011a) argues that such interfaces are better suited to collaboration than both single user computers and interconnected musical networks, as the ‘interaction with digital data is more embodied and social’. While this statement may have merit, Xamb´o et al. (2011a) does not support this claim though a direct comparison of the collaborative affordances of touch tables and other interfaces for musical collaboration. Although the work in this thesis does not deal directly with touch-screens, tangible interaction or tabletop displays, these projects are included due to the similarities they share with other aspects of the work studied in this thesis. In particular touch- screen interfaces support real-time co-located interaction, and multi-touch musical interfaces allow multiple users to simultaneously interact with, and contribute to a shared musical or sonic product. The spatial and territorial aspects of table interfaces are also reflected in the interface designs used for the studies presented in later parts of this thesis.

Another crucial consideration is the physical orientation and configuration of the shared in- terface, and how this supports group interaction. The reacTable was deliberately designed as a circular table, so as to create a situation where there is no ‘head position or leading voice, and with no privileged points-of-view or points-of-control’ (Jord`a et al., 2007). While not discussed in his publications, photographs of Davidson’s interface show it in a diagonal orientation, rather than as a horizontal table-top, and the multi-touch platform is described as a ‘drafting table’ (which is typically tilted, rather than flat). This has implications for how a group of people might interact with and orient around the interface. A slanted table clearly privileges those stood facing the front, and many of the graphical widgets (e.g., the on-screen musical keyboard) privilege a user who is facing them in the correct orientation, and although the widgets could potentially be rotated and re-sized for use by someone standing at a different side of the drafting table, they may not promote the same degree of collective interaction as the reacTable pucks which are in- dependent of orientation. Finally, the physical nature and arrangement of the reacTable pucks presents all users with a persistent account of the system state and makes changes immediately visible. On other hand, on-screen widgets could potentially become minimised or obscure each

other, therefore hiding certain information about the system.