Harmonic Walk Application Framework

Table 5.2 lists all the data that describe the Harmonic Walk application with respect to the con- ceptual framework reported in Sections 4.2 and 4.3. The musical information, geometric inter- pretation and spatial positioning have already been discussed in the previous Sections. In the Harmonic Walk application the interaction mechanism is very important because all the knowl- edge the user acquires is expressed by her/his movements direction and timing. In the first application’s activity the user has to follow the straight path which represents the subsequent harmonic regions. S/he has not to worry of where to go, because the regions are put one after the other, but rather has to focus only to when to move. To obtain this information the user has to store in her/his memory the song chosen for the activity and apply on these data her/his pre- viously acquired implicit knowledge of the harmonic changes. As can be seen in Figure 5.11, a

56 Interactive Spaces: Models for Motion-based Music Applications

Figure 5.9: Harmonic Walk’s system architecture. The RGB video data are processed by the three Zone Tracker modules (background subtraction, morphological image processing and blob tracking). The barycentre position is compared to the stored masks and, through the application’s musical presets the audio output is produced.

musical period offers to the listener various levels at which s/he can synchronize her/his move- ments. Whereas the period, phrases, semi-phrases and bars levels have a regular timing, the harmonic regions have not always the same duration. Moreover the harmonic region durations overlap the phrase limit in the middle of the period. Thus, the choice to coordinate the step to the harmonic rhythm level (selective entrainment) is not trivial and has a great importance from the cognitive point of view, because it means that the user is aware of the musical element that is the object of the activity, and that s/he has recognized its features. As the songs have words to mark their rhythmic events, some syllables occur also in the harmonic change points. Thus, for reasons of practicality, the syllable which mark the harmonic changes are called the “syllables of change”. The syllables of changes are very important to help the harmonic change points iden- tification, memorization and assessment. The exploration activity is characterized by freedom in the movements inside the six zones of the circular ring corresponding to the musical chords harmonic space and by freedom in the timing of this exploration. In this phase the user has the possibility to listen carefully to the sound of the chords and to link this perceptual experience to the various zones of the circular ring. Thus, a cognitive map of the harmonic space useful for the melody harmonization is built. The user is free of space and time constraints when exploring the environment. The observation of her/his behaviors are an interesting source of information about the user’s belief and feeling of the harmonic space. The ultimate melody harmonization activity puts together the time constraints of the harmonic rhythm detection and the chord localization

Chapter 5. Responsive Floors Music Applications 57

Figure 5.10: Two masks for floor surface division in the Zone Tracker application. A 25-squares zone partitioning and a circular ring with six active zones, designed for button interaction. Blue areas are activated by a user’s presence.

HARMONIC WALK APPLICATION FRAMEWORK Aims Discovery of the

melody harmonic rhythm Exploration of the harmonic space Melody harmoniza- tion Interface Musical Information Melody segmenta- tion

Tonnetz representation of a major tonality harmonic space

Geometric In- terpretation

Sequence of melodic units along a straight line

Circular ring with three main and three par- allel roots

Spatial Posi- tioning

White crosses along the surface borders

Six black crosses put in circle at the centre of the floor

Interaction Where to move

Along the straight line

Free choice inside the circular ring

Follow the chord se- quence which fits the melody

When to move Selective entrain- ment with the harmonic rhythm

Free timing Selective entrain- ment with the harmonic rhythm

Table 5.2: The table shows Harmonic Walk aims, musical information, geometrical interpre- tation and spatial positioning of interactive landmarks employed for each activity. The double vertical bar divides the discovery of the melody harmonic rhythm activity because it employs a different musical information and geometric interpretation than the other two activities.

58 Interactive Spaces: Models for Motion-based Music Applications

Figure 5.11: Formal organization of a typical music period with phrases, semi-phrases, bars and harmonic functions and regions rhythm. Between the second and the third semi-phrase an overlapping harmonic region is pinpointed.

stored in the cognitive map built during the exploration activity. In this activity the embodied knowledge of the harmonic framework of the melody is expressed through the sequence of the positions corresponding to the musical chords and through the timing of these movements. The user play-acts the harmonic sequence and behaves as if s/he actually “is” the harmonic sequence. Thus Harmonic Walk may be considered as belonging to the category of play-acting games dis- cussed in Section 3.2.3.