Artificial Music
A review of the use of Artificial Intelligence and Artificial Life in Music
By
Dr. Jonathan P. Wakefield
Department of Engineering and Technology School of Computing and Engineering
Brainwaves
• Ideally would like a machine that can convert imagined music into audio
• IBVA (Interactive Brainwaves Visual Analyser) is a system that can map certain EEG
Sound Design – SST design (1)
• Ricardo A. Garcia has undertaken work in
automatically designing Sound Synthesis
Techniques (SSTs)
• Basically he has a target sound he wants to
synthesise
• Views design as a search of a huge
multi-dimensional SST space
Sound Design – SST design (2)
How does it work?
1. Produce a population of random topologies
2. Then uses mathematical optimisation techniques to determine parameters e.g. filter cutoff
3. Each candidate solution is evaluated using a fitness function (error metric)
Sound Design – Exploring Sound
Space (1)
• Hardware and software synths are generally
hardwired with a particular SST e.g. subtractive, additive, physical modelling …
• To generate useful and interesting sounds with a new SST a user has to go through a learning curve • James Mandelis has addressed this problem with
his Genophone hyperinstrument – it allows users to perform sound design without understanding the underlying form of synthesis
Sound Design – Exploring Sound
Space (2)
•
How does it work?
1. Start with population of hand crafted sounds 2. User evaluates each sound (parameter set) 3. User then selects which sounds (parameter
sets) s/he wants to use as parents
4. Selected parents generate new sounds (parameter sets) by reproduction and mutation
Performance Mappings (1)
• Controller assignments also need knowledge of
SST to map controls to useful combinations of SST parameters
• Mandelis’ Genophone also allows the evolution of performance mappings
• This is carried out in the same way as the evolution of synthesis parameters and carried out at the same time
Performance Mappings (2)
• These are interfaced to 5 control knobs on a
Korg Prophecy synth to make realtime
changes to sound
• Each controller can control 4 parameters
• This allows local exploration of the
soundspace with the previous “Sound
Computer-based DJ (1)
• Dave Cliff of Hewlett Packard ) has developed a DJ computer system that sequences (i.e. chooses tracks and in what order) and mixes (i.e. beat
matches and crossfades)
• In 2000 played off against a DJ in a club for New Scientist (45 out of 72 clubbers spotted the
Computer-based DJ (2)
• In 2001 made a more sophisticated version
• Clubbers wear wristwatches to provide feedback.
Monitor their location, heart rate, perspiration and an accelerometer monitors activity and communicate to computer via bluetooth
• Splits songs into individual tracks eg. drums, bass, vocals, keyboard hooks.
• HPDJ picks individual tracks and overlays them. • Uses GA to evolve good music with clubbers
Composition – ATNs (1)
• David Cope has a piece of software called EMI (Experiments in Musical Intelligence)
• It is derived from Mozart’s Dice Game but is much more advanced
• Most importantly it doesn’t have a single fixed phrase template
Composition – ATNs (2)
• How does it work?
– Human decides on a set of example pieces for EMI to analyse
– EMI searches through these pieces using a pattern-matcher to find recurring templates of significant length.
– EMI also builds up lists of all the alternative fragments which can fit each slot in a template.
Composition – ATNs (3)
– The ATNs represent valid musical sequences in a particular style and are used to generate music in that style
– Final stage is pattern matcher which extracts signatures from examples and then adds
Composition – ATNs (4)
• Does it work?
– Produces convincing pieces in a composers style
– Compared to a lesser human composer trying to mimic a master
– Cope says “usually lacks the true spark of genius”
Composition – Markov Chains
(1)
• Markov Chains are good at representing short term musical patterns
• But have problems generating convincing complete pieces
• Continuator, developed by Francois Pachet exploits Markov chains’ good points whilst avoiding its bad points
Composition – Markov Chains
(2)
• Musician organises pieces high level structure while Continuator “fills in the gaps”
• Bit like a much more advanced version of an arpeggiator or auto-accompaniment system
• Automatically learns and imitates of musical styles and the music it generates is stylistically consistent • But it is also a new kind of “instrument” that can
Composition – Markov Chains
(3)
• How does it work?
– Continuator receives MIDI from musician – It segments MIDI into phrases
– Analyses phrases and builds up Markov model – At same time, after a phrase is played in by the
musician, the continuator generates a
continuation based upon the Markov model – The generated continuation is output as MIDI
Interactive Composition –
Markov Chains (1)
• Instead of just using learnt Markov probabilities to decide which of alternative continuations to play can take into account notes currently being played to take account of harmony
• Prob(x) = S*MarkovProb(x) + (1-S)*(NoNotesInLast8 / 8)
• Varying S from 1 (automaton) to 0 (probability totally based on input) gives different output
Interactive Composition –
Markov Chains (2)
• Does it work?
– Can produce a stream of notes where it is
usually not possible to tell what was played by the user and what was played by the
Continuator
– Aha effect when musicians hear it echoing back something they played earlier or realising it is starting to play in their style
Composition – Cultural
Approach (1)
• Can evolve music using GAs using human as
fitness function but this is very time consuming – can replace with a computer critic but this hasn’t been very successful so far
• Cultural approach uses GAs and individuals socially interact with their music
• Note: Music is meaningful to their world but not necessarily ours
Composition – Cultural
Approach (2)
• Todd and Werner – coevolved male
composers and female critics
• Composers have 32 note tune (from 2
octaves)
• Critics have expectations encoded as 1
storder Markov chain
Composition – Cultural
Approach (3)
• How does it work?
– Composers initialised with random tunes – Critics initialised with folk-tune melodies
– Each critic listens to a number of randomly selected composers and selects one to mate with based on her Markov chain
– Mate (and mutate) to produce one new child per pair with randomly chosen sex
Composition – Cultural
Approach (4)
• Eduardo Miranda has developed a mimetic
model
• Each agent stores its sound repertoire and
other parameters in memory
Composition – Cultural
Approach (5)
• How does it work?
– At each round agents pair up and …
– First agent plays a randomly chosen tune from its repertoire (if rep is empty plays random tune)
– Second agent finds most similar tune in its rep
– First agent then compares the returned tune to its rep. – If original tune is most similar then second agent will reinforce the existence of the tune it sent out and also try to modify it to be more like original
Computer critic (1)
• Hit Song Science is a piece of software by Polyphonic HMI of Barcelona that can determine whether a song is likely to be a hit record
• Software looks for underlying mathematical patterns in music
• Use a hit database of 3.5 million songs from last 50 years.
Computer critic (2)
• They say that if you look at songs from just last 5 years, they are clustered into a limited number of small groups spread across the “universe”.
• If you want a hit you need to position your song in one of the clusters.
• What about somebody new and original? The next big thing? Using the above just makes all music end up being the same? They say this is NOT true
Track Recognition - Shazam
• Proprietary pattern recognition technology
(patent-pending) that can identify recorded audio even under noisy conditions (in 30 seconds) and send song and artist back as SMS message.
• Database contains over 1.7 million tracks, and is growing with another 5,000 or so every week, covers UK and German markets.
