Sources of Controls for Parameter Mapping: Direct and Indirect Control As mentioned above, the required controls can be classified into two different

categories: direct and indirect. The drawback of direct control is that the performer has to engage in operative gestures which can interfere with the performance activity. The dichotomy of intentional changes to the system and interruption to the musical flow could only be resolved by using an assistant. This is common practice in compositions combining acoustic instruments and live electronics330 _{as well as in approaches in which} the computer recycles audio feeds from acoustic players331_{. The results of performances} yield some musical interest, but nevertheless reveal a fundamental dilemma about electronic real-time processes that are based on material captured from the live acoustic performance as opposed to systems involving synthesis and pre-sampled material. In the absence of a predetermined structure outlining the essence of the musical content or characteristics, the performer operating the electronics has no distinct musical identity. The apparent endless electroacoustic possibilities to manipulate the captured live

328 _{Emmerson 2007, 138.}

329 _{Bown and Lexer 2006. See Section 4.6 for more detail.} 330 _{E.g. Harvey: Bird Concerto with Pianosong (2001).}

331 _{Solar Winds by Lawrence Casserley and Evan Parker (http://www.touchmusic.org.uk/catalogue/}

to35_evan_parker_lawrence_cass.html), Frédéric Blondy and Diemo Schwarz, Concert on the 18.11.2011 at The Warehouse, Waterloo, London. This has also been investigated by the author using the piano+ system on occasions when no piano was available: e.g. Seymour Wright as part of the Foldback Festival, 295 Haggerston Rd, London E8 on the 05.08.2006, with Durrant, Drew, Lely and Mattin Shunt Concert: Man & Machine 14.06.2007.

material has can nevertheless rarely avoid references to the source. The electroacoustic material reworks musical ideas of the acoustic player but retains melodic and motivic contours, rhythmic arrangement and the spectral distribution of the harmonics in the sound. The electroacoustic part can fill the gaps the acoustic performer leaves, but the musical material will hardly develop true autonomy in content to take lead during the performance. Also the acoustic player is in a peculiar situation when their material persists beyond their contribution. They are aware that their sounds form the basis of the processes and might be able to develop strategies to ensure specific material appears in the electroacoustic textures, but they are powerless to remove material from the electroacoustic process.

4.2.1. Indirect Control Through Audio Analysis

Audio analysis tools are intrinsic to the augmentation of the acoustic piano. Tools measuring amplitude332_{, pitch}333 _{and significant changes in the frequency spectrum}334 _or registering the percussive attack of a new sound, are a gateway to creating music/sound aware processes. The piano is in general well-suited to audio analysis: notes played conventionally have a clear attack, pitches are clearly distinguishable and stable, and it can produce a wide range of tones and timbres. There is the potential to design processes that, for example, only happen if a sound louder than a specified value, a particular note, or a new onset (attack) is detected. Precise analysis would require monophonic lines, as the process cannot distinguish polyphonic voices. The analysis treats the input as if it is monophonic: estimating one fundamental for the entire frequency spectrum. Musical and aesthetic rules have to be encoded and implemented. The machine is neither listening nor capable of making decisions equivalent to those of a performer, unless modes of listening and responding were modelled335 _through appropriate algorithms and implemented by the programmer, as simple or sophisticated as the rules might be.

332 _{Available in Max/MSP: peakamp~ enables amplitude readings of the audio.}

333 _{Available for Max/MSP: fiddle~ analyses audio spectrum to estimate the fundamental of a note (http://}

crca.ucsd.edu/~tapel/software.html), several other algorithms are available within the Max community (e.g. sigmund~,pitch~, analyzer~), as well as from the IRCAM real-time musical interaction library (http://forumnet.ircam.fr/704.html?&L=1)

334 _{Available for Max/MSP: bonk~ onset detection (http://crca.ucsd.edu/~tapel/software.html)}

4.2.2. Indirect Control Through Sensor Readings

Experimentation with sensors is another important part of this research336_{, as a means to} derive control data from musical gestures and to enable control without needing to operate the laptop or MIDI controllers placed near the piano manually. Pressure sensors337 _{can be placed unobtrusively at various places on the frame close to the area of} activity, making them easily accessible to control processes directly. When attached to objects or tools (e.g. beaters) used to play inside the piano, the sensors can capture data about the physical intensity of activity (i.e. strength of grip) for indirect control. While this method has great potential, the required cables for the sensors can make their use problematic. Affordable practical wireless solutions have not been available. Infrared (IR) distance sensors338 _{have the advantage of observing movement in the air but can} become unreliable in certain light conditions. Taking inspiration from Waisvisz’s The

Hands339 _{or the commercial P5 Glove}340_{, the placement of sensors on the hand (e.g.} bend or pressure sensors) give flexible control opportunities, but restricted performance activities on the piano, even when reduced designs used only one IR distance sensor at the wrist and one bend341 _{and pressure sensor on the index finger}342_.

Another successful approach uses video colour tracking. A brightly coloured velcro343 ring at the thumb allows flexible two dimensional control which has some advantages to the IR sensors. The ring being easily visible for the camera, or hidden by moving the thumb below the palm of the hand, enables a more intentional binary level of control, whether the data gathered is being used or not344_{. More complex routing options allow} the tracked movement to be used to query a picture for colour values345_{, which are used}

336 _{First experiments were using the Pocket Electronic by Doepfer (http://doepfer.de ), the Gluion}

(Barefoot) (http://glui.de ) is the current digitising board for the sensors used in my setup.

337 _{Interlink Electronics FSRTM 400 Force Sensing Resistor.} 338 _{Infrared Proximity Sensor Short Range - Sharp GP2D120XJ00F.} 339 _{Waisvisz 1984, 1989, 2005 http://www.crackle.org/TheHands.htm.} 340 _{Vrealities http://www.vrealities.com/P5.html.}

341 _{Flex sensor 2.2" by Spectra Symbol.}

342 _{Experiments conducted, see Appendix V – piano+ (version 2005) Figure B9.}

343 _{The loop side of a velcro strip shows a very even colour throughout, as it does not reflect light or casts}

shadows.

344 _{Videoed example Interlace 27.03.2004.}

345 _{This system was developed for the development of Emmanuelle Waeckerles VINST project. http://}

to determine parameters of processes346_{. A more complex system of a three-dimensional} colour-mediated control structures was developed, consisting of colour-mediated audio analysis data347 _{and a three-dimensional ‘data-space’, and applied in concerts.}

Such video based gesture analysis implementations, however, proved too impractical and CPU intensive to form a viable performance system348 _{and are therefore not} discussed in further detail within this thesis as they have not been used in performances. The current setup of the piano+ uses the Gluion board to digitise the data from a variety of sensors. Two tilt sensing acceleration sensors349_{, the first attached to a small metal} ruler to enhance the perception and control of the tilting gestures with an addition touch sensing strip350_{. The second is mounted on a separate breakout circuit combined with a} gyros sensor351 _{and a pressure sensor. Both devices}352 _{are intended to be held and used} with the hand or rested in appropriate angle on the instrument. Two IR short range sensors mounted on a small breakout circuit are usually placed inside the piano to allow sensing the distance of the hands, arms, or even head placed above the sensors.

4.3. Comparative Case Studies