Disembodied Voices Application Framework

Also if the conductor in Disembodied Voices acts both as a composer and as a performer, the application’s musical information and geometrical interpretation concerns only the expressive interaction function. The Disembodied Voices composition algorithms structure is not open to interaction. Thus, no geometric representation is necessary, because the performer can only

Chapter 6. Three-dimensional Spaces Music Applications 83

DISEMBODIED VOICES APPLICATION FRAMEWORK

Aim Expressive performance of algorithmic compositon

Interface

Musical Informa- tion

The musical conductor model Geometric Inter-

pretation

User-centered emispherical space

Spatial Positioning

Interactive score production Expressive performance Cue button at the forward-highest

Stop button at the rear-lowest Emispherical regions partitioning

Interaction

Where to move To the cue/stop button position

Score alignment phase: to the appointed space region Expressive alignment phase: to the best sounding position

When to move When the change to the next section is felt necessary

Score alignment phase: in due time with musical score directions

Expressive alignment phase: whenever an expressive change is needed

Table 6.2: The table shows the Disembodied Voices aim, musical information and geometri- cal interpretation. From the spatial positioning on, the table is subdivided following the two application’s tasks: interactive score production and expressive performance.

decide the duration of the various sections of the score but cannot change the musical structure of the event. Thus, the difference between interactive score production and expressive performance features is present only at the spatial positioning level, where one button for cue attack and stop are positioned at the forward-highest and rear-lowest positions respectively. The interactive regions of the Disembodied Voices interface are delimited by some thresholds positioned on the sagittal (azimuth), frontal (radius) and longitudinal plane (zenith). Unlike the active floor applications interfaces, where the user’s attention focuses on the area of the various interactive landmarks, in the three-dimensional space the attention focuses more on the thresholds than on the regions. The main reason is that in Disembodied Voices’s three-dimensional space the active regions are not visible to the user. The characteristics of musical performance employing these invisible regions are explained in the next Section 6.1.7.1.

6.1.7.1 Performing on a no-touch Instrument

An important feedback when playing a musical instrument is the haptic sensation related to the mechanics of the instrument. This feeling is divided into: tactile (pressure, surface curvature, orientation, moisture, friction, etc.) and proprioceptive and kinesthetic sensations (Rovan and Hayward, 2000). They are so important to be considered as part of the instrumental performance. For example, they allow the players to orient themselves in taking positions, to improve tuning based on simple auditory feedback, to facilitate articulation, phrasing and thus expressiveness, to improve music pieces memorization (via the retention of certain situations resulting from haptic gestural sequences), to perform complex compositions or to read them at sight focusing the eye

84 Interactive Spaces: Models for Motion-based Music Applications

Figure 6.5: Visualization of an example of the musical result of Disembodied Voices compo- sitional algorithms. In particular, starting from the left, an A (440 Hz) pedal, four short A at different octaves, and a slide towards a very low A.

on the score rather than on the instrument, etc. The characteristic of the no-touch instrument is to communicate with the computer through gestures without being in direct contact with the control surface. As a consequence, the haptic feedback in these performances is totally lacking. Rovan and Hayward (2000, p.300-301) accurately describe the performance process under these conditions, trough the following steps:

1. the performer, who is more or less forced by the system, must have in his/her mind an executive idea (intention)

2. then s/he must run the gesture to perform the intention. Through the control surface (which is imaginary and of different sizes and shapes) the gestural data become sound

3. evaluate her/his action through the vision and the kinesthetic sense 4. hear the resulting sound

5. amend her/his acts through the vision and proprioception 6. change her/his intention by appreciating the resulting sounds 7. and so on ...

It’s obvious that in this scenario the lack of haptic feedback increases the importance of the visual and proprioceptive channels. The first Max/MSP patch is not only used for gestural data processing but also to produce a visual feedback of the interaction. For this purpose various GUI objects of different sizes and colors have been used to display the current state of the interaction in order to make it clear even far away from the screen. The second patch produces a visual

Chapter 6. Three-dimensional Spaces Music Applications 85

Figure 6.6: Nine stages of a Disembodied Voices score noted in summary form for performer’s reminder. Pitch events (la lungo, long A), dynamics (p, mp, etc.), ring modulation effects (MP, P, etc.), frequency shift (∧) and second frequency shift (+) are displayed.

feedback of the result of the compositional algorithms by using a lcd object where small spheres display the sounds produced by these algorithms. On the x axis is represented the time, on the y axis the pitch (see Figure 6.5). The problem of all these visual feedbacks is that they are pro- duced retrospectively, just after the action has been concluded or the position has been reached. They have only a confirmatory function of the intention once it has been properly completed. Unfortunately, they do not have any predictive function with respect to a possible mistake. In the case of Disembodied voices a real time three-dimensional display of the interpreter’s avatar could be useful, as well as the graphics of the thresholds in the three-dimensional control space. The Max/MSP patch 1 also displays instructions for the performance of each cue. These notes act as a reminder for the conductor and include current event description as well as hints about the expressive interaction to be performed, as shown at Figure 6.6.

6.1.7.2 The Disembodied Voices interaction paradigm

The interaction paradigm of Disembodied Voices is subdivided in two sections depending on the different performances tasks (interactive score production and expressive performance). Infor- mation about both tasks is contained in a score which is organized in various subsequent stages. Every stage contains some message for the compositional algorithms and some directions for ex- pressive performance. An example of notes employed for recalling the score requirements to the user is shown at Figure 6.6, where pitch events (la lungo, long A), dynamics (p, mp, etc.), ring modulation effects (MP, P, etc.), frequency shift (∧) and second frequency shift (+) are noted in summary form. As far as concerns the interactive score production, the user is free to decide when to trigger to cue button and hence to progress to the next score stage, or when to stop the performance. The expressive performance section is subdivided in the score alignment phase, where the user changes the active region following the score directions, and in the expressive alignment phase, where the user, once has reached the active zone, moves inside it searching the

86 Interactive Spaces: Models for Motion-based Music Applications

best musical result. In this case the musical gesture is dependent on the expressive intentions of the user and acts both as the cause and the effect of sound production (Cadoz et al., 2000). In this way action and perception are linked in a continuous loop that drives the user’s movements and timings.