Saxify system

Over 18 months of research was committed to developing the Saxify software. A first version was developed in the Java programming language. It was subsequently decided, based on superior data handling characteristics of the R statistical programming

language, to abandon the Java version. The R Studio product (version 1.0.143) was used for development, and all of the code has been verified in R language version 3.5.0 nicknamed Joy in Playing. The software involves four major parts – Saxify.R, Work.R, Section.R and Plotter.R.54 _{Saxify.R is the control program. It directs two other}

programs, Work.R and Section.R, to perform specific tasks (described below) under its

53 _{Valdimir Levenshtein, ‘Binary Codes Capable of Correcting Deletions, Insertions, and Reversals’,}

Soviet Physics Doklady, 10.8 (1966), 707–10.

54 _{Please see complete listings of the software in Appendix B. Note that the code is released under a}

Creative Commons license that allows anyone to use or modify it subject to the limitation that no commercial use may be made of the code nor of any derivative versions.

122

control. Work.R is a class that defines a musical work. It consists of multiple sections, each defined by the Section.R class.

Plotter.R performs unsupervised machine learning on a Dissimilarity Matrix, using the PAM (Partitioning Around Medoids) technique. We might want answers to a question such as ‘What are the six best groups we can make out of X’.55This type of data analysis

is important to the Saxify procedure since it facilitates visualising patters in music data that are not readily evident from the tables of numeric data.

It can also generate Silhouette Diagrams to show the strength of a particular PAM scheme and generate Elbow Diagrams to assist the decision as to how many clusters are in the DM data. The PAM technique—k-Medoids clustering—was chosen because it has several advantages over conventional k-Means clustering. It is more tolerant of outliers, and it can take a dissimilarity (or even a distance) matrix as direct input. Following a data processing run, Saxify produces a control report entitled Performance

Analytics as shown in Figure 4.17. It shows the work processed (‘Schoenberg

Phantasy’), date and time of the run, number of performances processed, identifier of

each performance, calculation of DMs (two separate variables, and combined), section breakdown (two separate variables and combined). Then it reports the Dissimilarity Matrices being calculated, followed by a repeat of this process for each section of the work. Finally, there are some runtime statistics, the most significant being the Elapsed Times for this run (in this case just over seventeen seconds). The performance data (including the PNT) is read from the Excel (or other) data store. Then the software calculates three DMs, one for each variable and one for the variables combined. Finally, a series of three similar DMs are calculated for each section of the music as specified in a map. The DMs are all stored in pre-defined folders:

Schoenberg Phantasy, op. 47 Performance Analytics Tue Jun 5 14:12:03 2018

---

Process performances data from Excel - (44 performances) ...

PNT, Koldofsky-Steuermann51, Varga-Krenek51, Kolisch-Stock53, Kolisch- Willmann53, Kolisch-Willmann54, Bress-Reiner62, Baker-Gould64, Kolisch- Johansen65, Menuhin-Gould65, Goldberg-Webster66, Kolisch-Johansen66, Gross- Grayson74, Gotkovsky-Gotkovsky76, Kremer-Maisenberg77, Kagan-Lobamov78,

123

Veselka-Dratvova85, Arditti-Litwin94, Kremer-Maisenberg94, Israelievitch-Lemelin00, Lang-Lang00, VanDerMeer-Grotenhuis01, Walch-Korber01, Wallin-Pontinen01, Huang-Vainstein02, Koh-Uchida03, Bellini-Gianotti04, Kim-Bowles04, Repin- Lugansky04, Doll-Marton05, Borup-Ernst06,

Widmann-Lepper06, Frayanova-Pashchenko08, Colbentson-Appel10, Moller-Oquist10, Revich-Kolodenko10, Schulte-Oldfather10, Yamada-Kynoch10,

Anderegg-Funderburk11, Fukuhara-Roden11, David-Gompper12, Soltau-Auvil12, Swenson-Chung12, Degand-Peyrebrune13, Felberg-Gordon13,

Calculating performance Dissimilarity Matrix...3/3 Calculating performance Dissimilarity Matrix...2/3 Calculating performance Dissimilarity Matrix...1/3 Section - 0: A, Grave I #: 1 of 4 Variable: 3...2...1... Section - 1: B, II #: 2 of 4 Variable: 3...2...1… Section - 2: B', ii #: 3 of 4 Variable: 3...2...1... Section - 3: A, III #: 4 of 4 Variable: 3...2...1... ...Run ended

User System Elapsed 17.338 0.702 17.284

Fig. 4.17 Run control report 4.15 Data storage

Figure 4.16 shows a useful folder/directory structure that is recommended to hold the input and output data. Individual items are described below.

Fig. 4.18 Directory/Folder structure

_map.csv defines the sectional breakdown of a work. It contains entries that define the structure in terms of its start and end bars. _PNTCalculator.xlsx holds input data in an

124

Excel worksheet that is presently used to import and store all performance source data including the Performance Norm (which is automatically calculated on the worksheet from the other tabs). Saxify is notified where the relevant variables are stored (as columns on the sheet) by parameters that may be set at run-time. Additional parameters include: the number of symbols to be used in the symbolic approximation step, frame size, weights to be applied to variables when calculating the combined values, and location (folders or directories) for DM outputs. _recordings.csv is a simple list of recording identifiers. OUTPUTn (where n is blank or an integer) defines a high-level

folder containing output data created by Saxify. It has a sub-folder, clusterPlots k=4, that holds output data files. K=4 signifies that the starting assumption for PAM is 4 clusters. K may be any value so long as it is notified to the Plotter.R program which uses the value as a starting assumption for the number of clusters.

This folder in turn contains a number of items. File ‘dmFULL ALL_clusters.csv’ is a file that holds the cluster identifier assigned to each recording. File ‘dmFull ALL.png’ is a realisation of the clustering for the specified variables combined. File ‘dmFull

Bpss_clusters.csv’ is the assigned clustering for the Bpss variable. File ‘dmFull

Bpss.png’ is the clustering graphic. File ‘dmFull dbDelta.png’ is the clustering graphic for the dbDelta variable. File ‘dmFull dbDelta_clusters.csv’ is the assigned clustering. These are followed by similar clustering details for each section of the music

performances as represented in file ‘_map.csv’ and numbered sequentially from Section0.

4.16 Chapter summary

Viewing large tables of numbers may be an unproductive way to detect patterns in data. Similarities in musical performance data may better be visualized using statistical

clustering. This chapter has described the Saxify analytical model which: (1) reduces a

set of performance variables to a symbolic string, (2) creates a performance norm to represent an average performance from a set of music performances, and (3) defines the quantum of dissimilarity between any two performances and between each performance and the PN. The distance between any two strings represents the degree of dissimilarity between the underlying two musical performances. It is calculated in a standard

statistical manner by looking up the distances between each pair of symbols in a

125

multiplying by the square root of the compression rate. Symbols that are one or two values apart (for example the combinations a, a or a, b) are treated as having have 0 separation. This helps to eliminate spurious dissimilarities that may arise from random noise in the data. Normal/Gaussian tables may be used to calculate the probability of random occurrence. In statistical terms, the acceptance region at 95% confidence is 1.96 standard deviations from the mean. We may conclude that values below 1.96 are not statistically different from 0. Values greater than 1.96 may be assumed to be statistically different at that 95% confidence level. Since the values in the PAA series were

constrained to have identical Gaussian probabilities of selection for symbolic

representation, the calculated dissimilarity measures—as expressed by MINDIST— are related to areas under the Gaussian distribution. High values of MINDIST may be assumed to represent real dissimilarity rather than differences that have arisen randomly.

This model will now be validated on sets of Chopin recordings while Chapter 6 will apply it to Schoenberg Phantasy.

126