Chapter 4: Technical Implementation – piano+
4.4. Parameter Space – the Potential of Triple Controller System
The piano+ patches have had various approaches to change controllers, implemented and tested over time. In many respects the most challenging work has been to develop a suitable approach. In hindsight it feels that choosing suitable effects processes was easy, while each control implementation affected performance practice significantly375. For example, an early version used drop down menus to change parameter control (i.e. from onset detection to manual control). This approach proved utterly impractical, as changes relied on operational activity on the laptop, requiring several clicks. By the time new selections had been made the musical flow was disturbed. In concert situations changes were therefore avoided, but one was left with the impression that the performance fell short of its potential as a change might have made the electroacoustic processes more appropriate to the situation and musically interesting.
The realisation of a dynamically changeable system, as proposed in the previous section, requires a sophisticated convergent control implementation, in which direct controls can merge with indirect controls by changing the weighting between the two control methods. In the scenarios given above, for example, this would mean that the manual trigger for granulation merges with the triggers from the onset detection: a sensor controlling the position between a set minimum and maximum value could be gradually replaced by values from the amplitude analysis.
To enable the combination of different controllers for parameter control (convergent mapping) all controller data streams are scaled to a range between 0.0 and 1.0, irrespective of their initial interfacing data376, data type and rate377. These are stored in audio buffers at discreet sample indexes. A direct MIDI controller (value range 0-127, received with associated controller number and MIDI channel) is converted into a <index> <value> list (e.g. ‘f35 0.3’), where the index is a symbol combining an
375 See Appendix IV – piano+ (version 2011) and Appendix V – piano+ (version 2005). 376 MIDI, continuous sensor readings from serial ports or network transfers, audio analysis data.
377 The development included versions using Max abstractions using send/receives and route, Jitter, FTM
(IRCAM) but these versions suffered from some sluggish response dependent on the overall CPU usage. The final version transforming all controllers into the signal rate (using buffer~, peek~, poke~, index~) has helped to overcome these shortcomings and made the overall system much more responsive and accurate. The difference was clearly felt and is thought of as potentially the turning point from a system which was interesting to use and for which one could see much potential, into one which brought everything together and enabled a real progress in the actual performance practice.
abbreviation for the controller device378 and controller number, and the value is the scaled value. For the example ‘f35 0.3’ the value ‘0.3’ would be written for sample index ‘35’ inside the audio-buffer ‘f’379 (Figure 4.5).
Figure 4.5: Scaling and storing direct controller values into the audio based control system
As an example for an indirect controller (Figure 4.6), the value derived from continuous controller streams through audio analysis (e.g. peak amplitude) would be scaled and assigned to the index ‘a0. Therefore the value received from the amplitude reading would be stored in audio-buffer ‘a’ at sample index ‘0’.
Figure 4.6: Scaling and storing indirect controller values into the audio based control system (e.g. ‘a0’)
The current system uses command line style definitions to set up to three controllers per parameter. The first is can be considered as the main controller while the other two influence the range of the first. For example (Figure 4.7), ‘a0 f35 f36 2000. 5000.’ would use the amplitude reading ‘a0’ to set parameter between 3000 and 5000 ms. The MIDI controllers ‘f35’ and ‘f36’ could adapt the given minimum and maximum settings 378 I.e. f0 - f65 for the controllers available from the Faderfox MIDI controller, a0 - a15 for data from
audio analysis, or g0 - g76 for the sensor readings from the gluion digitising board.
by 0 - 100%. f35 would enable to change the minimum between 3000 (R[min]) and 5000 (R[max]), f36 the maximum.
Figure 4.7: Parameter control module with command line style triple controller definition.
Mathematically the three controllers are combined within their range of 0. and 1.: V(c) = (v[a0] * (v[f36] - v[f35])) + v[f35]
This value is then scaled according to the range defined by the numeric range: V(p) = (V(c) * (R[min] - R[max])) + R[min]
Setting controller f35 to its minimum (f35 = 0.) and f36 to its maximum (f36 = 1.), the amplitude reading (a0) has maximum effect (values between 3000 and 5000) on the process. As the range is controllable by these two controllers it is possible to regulate how much the process is affected by the amplitude reading. This system has the scope to make a0 inactive and set the process to a fixed value in the case both controllers are set to the same value (f35 = f36). The range of action can be reversed by setting the maximum to a lower value than the minimum (f35 > f36).
To exemplify these relationships between the three controllers, a selection of scenarios (Figures 4.9 - 4.12) are described and a graphical illustration of its possibilities is given below.
In the first scenario two direct controllers are used to control the range of a control stream from the audio analysis as described above. Figure 4.9 shows this standard scenario where the data stream (C1) from the amplitude reading (a0) sets the parameter value within the range set by C2 and C3 (e.g. MIDI f35, f36), inclusive reversing minimum and maximum values.
full range reduced range fixed value reversed range
(a0) (f35) (f36)
0 % 100 %
Figure 4.9: Standard set. The range is shown here in percentage as actual values would be scaled according to parameter ranges in the processes.
Figure 4.10 shows a different controller set where C2 (i.e. amplitude envelope) and C3 (i.e. sensor reading) are alternated by C1 (i.e. MIDI controller). In this scenario the MIDI controller allows to change focus on either data streams (C1 or C1). C1 set to the minimum value would focus on C3, as the maximum value would focus on C2.
C1 focused on movement of C3 C1 focused on movement of C2 0 % 100 %
Figure 4.10: Changing focus between two continuous data streams (C2, C3) with C1.
Any other value of C1 would mix the controller streams is the resulting value would always be n% between the changing minimum and maximum values (Figure 4.11, i.e. C1 = 0.3 would take 30% of C3 and 70% of C2).
0 % 100 %
Figure 4.11: ‘Mixing’ two continuous data streams (C2, C3) with C1.
This triple controller set yields an astonishing practical use. With a minimum of operational tasks the behaviour of the process can be changed according to the musical situation and demand for intentional control. Focus on C2 would, for example, follow the amplitude reading of the performance activity. The performer (as well as fellow musicians) would in this case influence the parameter through their musical activity. Changing the focus on C3, allowing gestural control, would give independence of the current acoustic situation and enable the performer to have full control. It is considered here that the most powerful outcome of this setup is actually the mix between the two, so the performer has a degree of gestural control, while the acoustic situation has also its effect on the outcome.
The most extreme setup appears to gain contingent behaviour. In the case that all three controllers are set to continuous data streams (Figure 4.12) the influence through gestural and acoustic activity can be perceived but loses a clearly defined relationship. It hardly shows any accurately predictable results during performance as the interdependence between the three data streams is considerably complex.
0 % 100 %
Within their musical application the controls can be perceived as ‘tendencies’ and therefore have a heuristic character. The performer can explore these tendencies of control through musical gestures. Furthermore, this variety from loosely interconnected controller streams offers the possibility to use the same controllers in different combinations, to modulate several processes and effects simultaneously.