chapter four towards digitally-mediated creativity
4.2 systems of musical flow
4.2.1 from virtuosity to flow
The description of the creative user experience presented above shares characteristics Rittel and Webber (1973) outlined for “wicked problems” (Section 3.2), except that creative endeavours, in contrast to being “one-shot operations”, allow for and demand more experimental trial-and-error learning and practice. Outside of a live, public performance, musicians and composers can try new ideas without significant consequences, or the worry of failure.
feedback cycles In Section 4.1, support for rapid feedback cycles (H2) was
proposed as a way of supporting such experiential learning, and the development of computer music virtuosity. Leman (2008) expands on the idea of the feedback loop in music, identifying four repeating stages: play, listen, judge, and change (Figure 1). He uses this
‘action-reaction cycle’ as the basis for a philosophical framework for computer-mediated embodied music cognition, which he proceeds to use as an argument for the role of gesture in music interfaces. Leman argues that the approach has the power to afford “direct involvement” in music, by cutting out the indirection inherent in conventional interfaces. In many ways, his thesis can be seen as a musical reworking of embodied interaction (Dourish, 2004), drawing on the phenomenological approaches, pioneered by Heidegger and Husserl, and introduced to the wider HCI community by Winograd and Flores (1986).
Figure 1 Action-reaction cycle
(Leman, 2008)
‘readiness-to-hand’, transparency, and the “flow of work”
Karen Holtzblatt’s earlier work (on what later developed into
contextual design) similarly drew on this foundation, to discuss
Heidegger’s ‘readiness-to-hand’ as an approach to interface
transparency (see Section 4.1) – advocating “creative iteration”, the
maintaining of “workflow”, and avoidance of disruptions, in the user experience (Holtzblatt et al, 1988) – foreshadowing subsequent HCI rationales for Csikszentmihalyi’s own theory of flow (Norman, 1993; Bederson, 2004; Shneiderman et al, 2005).
virtuosity as a basis for “flow”
Flow, the theory of “optimal experience” (Csikszentmihalyi, 1988; 2000 – detailed in Section 3.7), describes a mental state that underlies creativity (Csikszentmihalyi, 1996) and enumerates several components (see Table 2-4) required to achieve it, for a given activity. In addition to advocating rapid feedback (H2), other
aspects of the definition of virtuosity in section 4.1 are implicit in some of the specific requirements of flow (see Table 3) – both call for interactions supporting focus, concentration, skill development and directness of control and feedback.
action-awareness
merging Of the two remaining components of flow (absent in Table 3),
action-awareness merging objectifies the resulting trance-like flow
state itself, where the individual is wholly-engaged in the activity, unfazed by external, environmental factors. It can thus be seen as the product of other components, such as loss of self consciousness and concentration and focus, as well as a corollary of distorted
sense of time.
intrinsic motivation
and rewards The other missing component similarly stresses a separation from
the outside world – that the activity be intrinsically-rewarding. Flow theory was developed upon the idea of intrinsic motivation, which occurs in an activity that is its own reward, in contrast to tasks requiring external incentives (extrinsic motivation; e.g. deadlines, penalties, money, recognition – see Section 3.7). An intrinsically-
rewarding activity is thus an enjoyable task that is both fun and
Table 3 – the design
heuristics for virtuosity, and corresponding components of flow
H1: Support learning, memorisation, and prediction (or “recall rather than recognition”)
Balance of Challenge and Ability
A learnable notation allows interaction skill to develop, and for achievement to scale with experience (see H3). By contrast,
traditional usability heuristics, such as Nielsen and Molich’s (1990), advocate “recognition rather than recall” (Minimize User Memory
Load) and minimal learning curves, tailored for novice use.
Clear Goals
Although final goals are hard to articulate, in creative endeavours, this component of flow concerns the moment-to-moment goals in interaction, and how easy it is for the user to discern the appropriate actions to take, to achieve a desired outcome. As such, it represents the latter stages of virtuosity, where mastery allows transparent use of the interface, balancing high levels of ability and challenge. Sense of Personal Control
A notation that allows skill to develop empowers the user.
Experience allows the user to predict what to expect so that they can plan ahead and actively drive interaction, rather than passively rely on visual cues and hints, allowing the program to set the pace. Loss of Self Consciousness
When an interaction is learnt or memorised to the extent it becomes automatic, it becomes reflexive rather than reflective, and the user becomes less consciously aware of how such actions and behaviour appear to others, allowing them to focus on the task itself.
H2: Support rapid feedback cycles and responsiveness
Direct and Immediate Feedback
Feedback should not only be fast, but direct from the domain. In a musical application, any change caused by an edit should be immediately reflected in the visual notation, and immediately available to audition, aurally.
Distorted Sense of Time
Slow or delayed feedback, such as those prompting idleness, wait prompts, progress bars or predicted completion times, implicitly or explicitly draw a user’s attention towards the passage of time. The user must be able to proceed at their own pace, in their own time.
H3: Minimise musical (domain-) abstractions and metaphors
Balance of Challenge and Ability
The use of simple interface primitives, which are easily learnt individually, provides novice users with a low starting threshold, while enabling increasingly greater challenges and functionality, when such primitives are combined (Resnick et al, 2005).
H4: Support consistent output and focused, modeless input
Concentration and Focus
Spreading interaction across multiple windows, views and input devices splits the user’s attention. Where modes or windowing are unavoidable, the user should be able to focus and concentrate on that area of program, without having to refer elsewhere.
flow in music
performance Musical performance is a commonly cited example of such an
intrinsically-rewarding activity (Leman, 2008), and can be used to illustrate many of the flow components (Csikszentmihalyi, 1998; 2000) and obstacles to be overcome. Acoustic instruments, for example, offer direct and immediate feedback, allowing experiential learning and, with experience, total, reflexive control (Williamon, 2004). Musicians can lose themselves in their instrument, for hours at a time (Boyd, 1992).
In the formative stages, however, a musician may have trouble articulating a pleasing sound, or achieving a sense of control. The lacking balance of challenge and ability can make them prohibitively self-conscious, especially if a suitably private practice space is not available. Similarly, music literacy, when taught as a pre-requisite to music interaction, presents an additional challenge for beginners that, as an example of “structured learning”, can detract from more playful and enjoyable “informal learning” experiences (Norman, 1993; see also Section 3.6), and deter students from persevering.
Ultimately, maintaining flow in the creative user experience requires protection from interruptions and distractions from the outside world, ensuring the user maintains focus, motivation and control.