Chapter 4: Technical Implementation – piano+
4.1.3. Second ‘Surrounding’ Layer – The Electronic Augmentation
Following this descriptive notation of the extended techniques, the technical design of the electroacoustic layer of the piano+ is now described in technical detail. The nature of the processes and their controls differ from the extended technique methods, because, depending on the parameter mapping, the controls are not necessarily based on a physical gesture. Although it is considered that a notation would be possible which might find useful application for various purposes, I have decided against it in this context. First of all the processes of the piano+ require an acoustic sound, therefore a method for an acoustic sound would be required as basis for coherent continuity. The processes manipulating either the vertical axis (frequencies) or horizontal axis (time) could reflect such changes. Figure 4.2 gives an example of how the sonic modification by a granulation effect could be visualised, illustrating how sections of the acoustic source continue as a slightly blurred and hazy extension of the sound.
Figure 4.2: Possible visual transcription of an electroacoustic process (e.g. granulation)
This example, solely given to attempt a visual continuity of the notation, is purely describing the aural result. Actual information about necessary performance gestures in relation to parameter settings (mapping) of the specific electroacoustic process is absent. If it were to display the parameter settings resulting in such a sound, then it would represent specific value curves of a specific granulator patch. An obvious path to generalise a scoring approach for the processes in coherence with those introduced for extended techniques is not apparent. The notation introduced for extended techniques established a continuum of physical gestures which can be explored in performance. The complexities and diversity in parameter mapping renders a similar approach for the electroacoustic augmentation as vague and too specific to particular setups and controllers and is therefore not pursued.
4.1.3.1. Infrastructure to Focus on Intended Functionality and Practicality
Instead, it is considerations of horizontal and vertical processes as established in Section 2.3.1 that have influenced the selection of processes included in the design of the electronic augmentation of the instrument. The fundamental motivation for my research was the potential of electroacoustic techniques to subvert particular sonic limitations of the piano. For instance the unavoidable decay of the piano notes can be overcome electronically using granular synthesis. The prominent attack phase of the piano can be eliminated with sampling techniques by applying fade-ins to disguise the attacks. The processes utilised within the system will be described later in more detail; however these two examples serve as an indication that the system has been developed to expand existing sounds rather than creating a system which adds additional musical parts intending to replace acoustic playing or fellow musicians with algorithmic processes315. From the start of the development of the system it has been a preoccupation that processes have to work in real-time without prerecorded and pre-edited material. This led to the dismissal of certain technology available i.e. the Disklavier, conventional sample players or sound design using sophisticated software packages. The exclusion of sophisticated devices and software tools developed in the wide range of electroacoustic fixed media compositions316 and accurate performance data acquisition through the MIDI protocol might be regarded as an omission with respect to sound-quality and control possibilities. But a system based on MIDI control input is too limiting in practical and logistical terms. The performance opportunities are very rare if one depends on the availability of a Disklavier and performance data acquisition would be restricted to conventional playing techniques, as no MIDI data would be transmitted when the performer engages in extended techniques, neither would the data reflect altered pitches of prepared notes. The alternative would be to include an additional keyboard like trigger system or to implement algorithmic processes to enable the use of prerecorded samples, which although explored at various stages through the
315 For example: a commercial software to replace ensemble musicians for practicing purposes is Band In
A Box (http://www.pgmusic.com/bbmac.features.htm), but also projects such as GenJam by John Al Biles (Miranda and Biles 2007, 137-169) and Voyager by George E. Lewis (Lewis 2000) result in the
replacement of acoustic players in favour of generative processes.
316 E.g. Trois études en duo (1991) and Huit esquisses en duo (1989) by Jean-Claude Risset for pianist on Disklavier with computer interaction. Jonathan Harvey’s Tombeau De Messiaen (1994) for piano and
development, was eventually dismissed as it contradicted the aesthetic approach and was considered a step too far into generative music composition.317
Consequently, piano+ was primarily developed with real-time sound modifications in mind and much research was directed to optimise parameter management, signal routing, and audio capture. An influential insight was that the type of controls used are far more decisive for the performativity of a computer-based system than the processes themselves. It is the inclusion of control methods that determine the musical paradigms convincingly discussed by Croft318. The paradigm of "backdrop"319 can be established with a minimum of control interaction, i.e. a fixed media piece or triggering suitable samples at specific points in the performance. The “instrumental” paradigm however requires strategies allowing immediate interaction and adaptation, to allow spontaneous changes to the musical situation.
4.1.3.2. Implemented Electroacoustic Processes
As mentioned in the opening of this chapter, the current design of the piano+ facilitates two granulation modules, a pitch modulator, a filter module and a partial synthesiser to supply sufficient means of vertical and horizontal sound modifications. Practical work and concerns about the exploration of the sonic potential of the instrument as a whole also influenced the final selection. This will be elaborated in Section 4.3.1 when the work and experiences with a live sampling module are described which has not been implemented in the recent design.
The granulation patches were personally built in Max/MSP using Granular2.5 by Sakonda320 as a model. The first granulation module321 allows to control start, length and volume of the grains as well as setting a random factor to modulate the playback position. The content is recorded in real-time, each recording records over the previous material, without clearing the entire buffer. The tail of the previous material remains available for playback, if the recording is re-triggered before filling the entire buffer. The grain speed results in possible transpositions. Minimum record time parameter,
317 Section 4.3.1 of this chapter will deal with live sampling in more detail. 318 Croft 2007.
319 Croft 2007, 62. 320 Sakonda 2001.
trigger prevention and overwrite are additions to help adjusting the behaviour of the granulator when the process is controlled by audio analysis.
The design of the second granulation module322 allows intensional accumulation and retrieval of material. Rather than replacing the buffer content, the recording sections are amended. An unlimited amount of sections can be recorded totalling a maximum of 10 min. Each section can be refocused and granulated in the techniques described above and mixing the sections is currently not possible.
The pitch module323 is allowing to cross-fade (parameter: swRingGizmo) between a ring modulator (parameter: carrier frequency) and a FFT-based real-time transposer (gizmo~, parameter: pitch). The high-pass filter is implemented to help to control feedback issues.
The fffb module324 uses a fast fixed filter bank (fffb~, n-number of resonating filters). This module uses 6 filter bands and allows frequency, Q and gain settings. Experiments had shown that the filter produces interesting ‘bubbly’ artifacts and glitches when extreme frequency, Q and gain values are used. This had sparked the idea that each filter bank to be enveloped (parameter: fffbRamp) so that the filter is only audible for a specified time. The frequency is set for each band sequentially and the range is controlled by two combined controllers (fffbPitch and fffbTranspose). The filter changes can be time limited (fffbSpeedLimit). The process required the implementation of several amplitude limiting procedures to avoid sudden and ear-splitting feedback in extreme frequencies.
The Partial Synth module325 uses frequency analysis on the module signal input. The raw frequency and amplitude data of the detected partials is fed into the resonators~ filter by CNMAT which filters white noise (or a separate audio signal) with these extreme filters. In this manner this module uses an approach to resynthesise the analysed signal.
322 See Appendix IV, Figure A6. 323 See Appendix IV, Figure A7. 324 See Appendix IV, Figure A8.
4.1.3.3. The Schematic of the piano+
The schemata of the overall software design (as used in the years 2010-2012) is illustrated in Figure 4.3. The grey components reflect the general structure of the performance system as described in Chapter 2326 while the black printed module descriptions show the implemented modules for the processes and control structure (see Appendix IV – piano+ (version 2011), Figures A1-A16 for screenshots of the Max patches and photos of the setup). The implemented structure reveals an emphasis on the sonic potential of the instrument, with the prime concern of enabling different timbres while retaining a haptic327, tactile, and heuristic relationship between the acoustic playing and the electronic manipulation. The alternative approach – aiming to implement generative algorithms for functional accompaniment and augmentation of solo performance to replace fellow players – has musical interest and validity, especially when considered as poles of a continuum which allows any combination of the two.
piano (Figure: A14) speaker system microphones contact microphones pickups extended techniques granulation (Figures: A5+A6) pitch processes (Figure: A7) fffb-filter (Figure: A8) partial-synth (Figure: A9) input routing (Figure: A12)
with dynamic effects and EQs for each channel
matrix-mixer (Figure: A10) audio analysis (Figure: A11) OSC controllers MIDI controllers sensors (Figures: A13-16) control-data-buffers (Figures: A2, A3 and A4)
acoustic instrument audio capture input routing processes output routing audio projection direct control indirect control sensors
triple controller system
piano+ flow chart feedback through
resonance board
feedback
For screen shots of the Max patch and individual modules, see Appendix – piano+, Figures A1 - A16
Figure 4.3: Schemata of piano+
acoustic sound waves acoustic feedback sound signal signal feedback
control signal (triple controller system) control data (direct and indirect controllers) control display update
326 Chapter 2.5 The infrastructure - General Structure of the Augmented Performance System
327 The haptic aspect is only in relation of the acoustic instrument here, which can however affect the
This continuum from the purely instrumental, where modifications, however subtle, are in direct control of the performer, and the completely generative, contributing an autonomous element in the performance that links all controls to data-streams from algorithms. In between, but closer to the instrumental, indirect controls derived from performance activity (e.g. sensors, controllers) and audio analysis328 would affect the parameters set by direct controls. Generative approaches could be added to these processes, for example influencing the effect processes over time according to algorithmic/analytical calculations. Closer to the generative still, we would find approaches which use generative algorithms329 combined with direct and indirect controls from performance activity (sensors, controllers) and sonic results (audio analysis, machine listening).
4.2. Sources of Controls for Parameter Mapping: Direct and Indirect Control