Future Work and Improvements

Improving Engagement, Usability and Usefulness

What changes could have been made it to make this experiment a success? The primary issue has been one of user engagement; there were simply not enough submissions to be able to begin to uncover the underlying geography of the fitness landscape. Currently the app is in a middle ground, it is does not offer fine-grained enough control to be a genuinely useful musical interface for a hobbyist or professional musicians. Conversely it is also perhaps a bit too complex and eso- teric for the average non-musician to fully understand and make use of. A future implementation should target either explicitly general crowds, and make the app more game-like, fun and easier to use, or conversely it should target musicians and music makers, offering fine-grains controlled over symbol to note mappings, and provide features that would allow it be used in a performative or recording context, such as being able to output MIDI or OSC to integrate the app with other studio tools. If the app became a genuinely useful musical tool, then it would be more likely to be able to reach a critical mass of use, and only then could enough data be gathered. It could also take the form of a VST/AU instrument for incorporation into common DAWs. Greater, more ex- plicit control over the musical outcome could make the users feel that they had greater ownership over their work, and be thus more willing to share it.

More Sophisticated Listener Model

An area of further work is to apply a more sophisticated listener model to the triangle. First-order Markov processes, with their very short term ‘memory’ are convenient because it is possible to extract exact information measures of redundancy and entropy rate. However humans have a more sophisticated long term perception of musical events, and as such a more sophisticated model could more accurately represent human perception of predictability and surprise. Further research is needed to determine what kind of information measures are needed for a more sophis- ticated listener model, such as one based on higher-order Markov processes. Wether it is possible to flatten these out into quality dimensions for the creation of a musical interface remains to be seen.

3.7. Discussion and Further Work 127

Alternative Interfaces, Data Gathering Methods

The triangular interface appeared to have issues of interface priming, and as such was likely polluting the gathered data. Alternative methods for gathering preferences with respect to infor- mation measures are needed if a map between these measures and aesthetic preferences is to be built. This could take the form of alternative interfaces free of priming (perhaps some sort of circular interface), or even A/B preference comparisons between pre-generated sequences might be more effective in gathering this data.

Application to Other Domains

An intriguing area of possible further work is to apply the Melody Triangle to other modalities. The triangle itself operates in the abstract, and as such could be applied to any domain, such as sequences of colours in video, or flashing lights. An exciting area of possible exploration is to apply it to the level of really short musical events, the domain of ‘microsound’(Roads, 2004). It could be used to control a granular synth for instance, rapidly triggering tiny snippets of audio to build sonic textures. Markov chain based granular synths have been explored previously(Miranda & Junior, 2005). It would be intriguing to hear what the phenomenal properties of the different areas of the predictability conceptual space correspond to in the timbral domain.

In this chapter the Melody Triangle, a musical interface derived from information theoretic properties of Markov processes, was presented. Its theoretical basis was outlined, including how it provides a mapping to a conceptual space of predictability. The Melody Triangle’s three in- carnations were detailed; an interactive installation, a desktop application and a mobile phone application. The interactive installation functioned well in its role as a communicator of research concepts to the wider public, however it was not practical for eliciting aesthetic preferences. The desktop version of the app, although demonstrating potential as useful performance tool and com- position aid, studies carried out with it in controlled contexts were also unable to identify patterns in aesthetic preferences. The mobile phone application allows users world-wide to submit their favoured settings, with compositions of notable worth uploaded by users of the app. Some over- all trends were identified, in particular a clear preferences for patterns of increased predictability. Additional identified trends however seem to be subject to issues of interface priming, and limits to user engagement were identified. This prompted the question of how these could be mitigated,

3.7. Discussion and Further Work 128

and some suggestions for this were presented as further work.

But could it be possible to bypass the issues of interface priming, by removing the interface from the picture entirely? Is it possible to not have computational generative process mediated by layers upon layers of mappings and devices, as the Melody Triangle is, but to be intimately embodied and explored in a natural way? Could the interface be made invisible, or even non- existent? These question inspired the creation of a novel system and art project, the subject of the next chapter: Keyebernates.

129

Chapter 4

Keyebernates

In chapter 2, some of the difficulties and challenges of parameter search in algorithmic composi- tion and design were outlined. One of the issues is that parameter-spaces are often vast, making an exhaustive search of the space not practical. Further little can be predicted about the outcome of a generative process from parameter values; similar parameters can yield very different results. The Melody Triangle embodied one approach to facilitating the search, by adding an inter- mediate interface layer mapping phenomenal similarity to parametric similarity. However such an approach cannot be applied to any generative processes, as it requires extensive knowledge and theory of the generative processes a-priori, and more often than not, this knowledge is not at hand.

However the difficulties do not end there. In traditional modes of creation, such as using a paint brush, there is an immediate and clear causality between action and output. As the artist applies paint to a canvas, instant sensory feedback enables her to continuously make evaluations of the work-in-progress. There is an embodied fluidity to this the process. The final result is the emergent consequence of the action-perception cycle playing itself out as it loops though the designer, her actions on the canvas, and her perception of the canvas. The designers’ sketches are the output of a “graphical conversation with the materials of design”(Schon & Wiggins, 1992), however in algorithmic and generative design, the free-flowing conversational nature of this interaction is difficult to maintain.

When composers and designers work with generative processes, the search through parameter- space commonly consists of discrete, discontinuous steps of parameter selection and output eval-

130

uations. In explicit generative creation, this discontinuity can manifest itself in the designer’s interaction with the substrate of the generative system, for instance while waiting for a computer to execute some code to render some media, or in the case of manual generative design, carrying out the procedures until it is possible to see the emergent results. Only then is the artist able to pass judgment on the work, and evaluate the parameter-selections or the design of the generative process itself.

This chapter introduces Keyebernates, an experimental system for control and navigation of the possibilities afforded by visual generative systems. Keyebernates was conceived to addresses the lack of fluidity and embodied control that is feature of common generative design methods. By using eye-tracking and gaze points as the modality of input, users navigate the parameter- space of a visual generative system with their eyes, the parameter values most gazed upon being reinforced, in a continuous and reactive navigation of parameter and solution space. When a viewer observes Keyebernates, they do not directly see changes as they happen. Instead by using the real-time data from an eye tracker, Keyebernates ensures that what is attended to does not change, and rather all change happens slowly and subtly in areas of the screen not currently attended to.

Like the other practical explorations in this thesis, a key aim of this study is to see if it is pos- sible to uncover the objective, inter-personal dimension of aesthetic judgments on a generative artefact; to find the elements that are common across individuals when value judgments are made on a parametric design. If it is possible to map out the parameter-space of a visual generative pro- cess with respect to some form of objective aesthetic desirability using gaze – and thus meet the criteria for ‘success’ defined in 1.2.1 – it could shed light on the cognitive mechanisms involved in making aesthetic judgments that are common to all individuals. Further, such a map could inform the construction of creative systems to assist in generating visual patterns that would be judged to be of aesthetic value by most people.

Keyebernates is both an experiment in HCI and experimental aesthetics, but also is an art

work in itself. Drawing explicitly on the metaphor of kybernetes - the steersman steering a ship across the chaotic forces of currents, from which the field of cybernetics draws its name - this experimental system interprets the perceiver’s gaze as a steering force against randomising chaotic forces.

4.1. Context 131

4.1.1. This is followed in 4.1.2 by an overview of previous work where eye tracking is used as control modality.

In section 4.2 the architecture of Keyebernates is outlined, describing in detail how the viewer’s gaze is used to navigate parameter and solution space. Section 4.3 then describes a user study with Keyebernates, where it was used with a very simple visual generative process consisting of overlapping circles.

The study was divided into two parts. In the first part, participants passively observed the screen, implicitly guiding the search through parameter space. As will be shown the users tended to pull the system towards similar areas of parameter space, suggesting that even within this very simple generative process, certain areas of the space yielded more visually compelling outputs then others.

In the second part of the study, participants were asked to actively search for a particular pattern. This was to test if volitional directed gaze, could be used to navigate towards appropriate areas of solution space. The results indicate that users are able to ‘will’ Keyebernates towards particular parameters with their gaze, however only across small distances in parameter space. As will be discussed, this is because the design of Keyebernates favours a slow detailed exploration of parameter space, and does not easily afford traversing large parametric distances.

In section 4.4 the results of the study are discussed, directions for further work are presented, and Keyebernates’s place and significance in a broader theoretical context is outlined.

4.1 Context