running the experiment

This section briefly describes the final preparations, launch and running of the experiment. Table 2 provides an overview of major events in this process.

2008 13 March _{reViSiT 0.92.1 released to testers, with data delivery code.}

17 October _{Experiment development begins.} 15 November _{Experiment website launched.}

reViSiT 0.95 Pro released to selected testers, with data collection code.

1 December _{Experiment begins (announced on website / forum only).}

reViSiT 0.99.1 Pro released to public.

14 December _{Experiment announced to mailing lists.}

reViSiT 1.0 Pro released, with full documentation.

18 December Experiment announced in Computer Music magazine (Issue 134).

2009 6 January _{iMPULS|IVE development begins.}

4 May _{reViSiT 1.1 Pro released, with user-definable keyboard shortcuts.} 23 May Over 1,000 experiment registrations.

6 September reViSiT 1.2 Pro released, with high-definition pattern editing.

20 December _{reViSiT 1.3 Pro released, with features for novices (e.g. mouse support).}

2010 9 May _{reViSiT 1.4 Pro released, with sample and instrument library screens.}

10 July _{Data received from over 1,000 users.}

26 December _{End of Experiment Questionnaire issued (see Chapter 9).}

Table 2 experiment milestones

5.5.1. testing experiment code

Testing of the data collection and delivery code ran in parallel with the testing of reViSiT Professional (see 5.2.4). In debug versions of the software, an additional console window is displayed, in which logged interaction events are printed as they happen, using Entry’s text() function. Log entries are created, encoded for saving, then instantly decoded for display, thus highlighting any problems in collection, encoding or decoding. After basic internal testing, the experiment code was integrated with the reViSiT Professional versions already being tested by selected users. This allowed a wider variety of interaction events and styles to be tested, as well as a wider variety of user systems, with different Internet connections and security (e.g. firewall) configurations. It also broadened testing to include data delivery mechanisms, which had largely already been proven, through their use in delivering reViSiT user feedback (see 5.2.4). As more data was collected, work began on the interactive visualisation

environment (IVE); designed to analyse the data, but also enabling

5.5.2. experiment launch

The experiment went live on 1st December 2008. Like reViSiT releases, the launch was staggered, to minimise the impact of unforeseen teething problems, with the program, registration process, and wider experiment system. As such, the initial announcement was only made through the website and forum, offering reViSiT 0.99.1 Pro – tested and complete, but lacking documentation, which was added over the subsequent fortnight. On the 14th, the experiment and reViSiT 1.0 Pro was announced to the 6,000 users on the reViSiT mailing lists, by which time the majority of issues had been addressed.

In November, a press release was issued to several online and print music technology publications, to catch their January issues, due for release mid-December. The announcement was carried by a number of websites, and appeared in the News section of

Computer Music magazine on 18th December, 2008. By the end of

the month, over 500 individuals had registered.

5.5.2. maintaining the experiment

The launch was followed by an initial surge in registrations, as the novelty of the software and experiment attracted press coverage and people found time to try the software during the holiday season, and as existing reViSiT users migrated to the reViSiT Pro. As the novelty wore off, so did the number of registrations.

At the same time, only about a half of registrants went on to provide data – others either failing the activation process (e.g. providing bogus email addresses), overlooking the need for suitable host software, or using systems away from the Internet. Furthermore, many registrants abandoned the program with only limited exposure. Although this was expected, the experiment objective was to observe users over time, as they developed skills with the program. Longitudinal information was assured from

reViSiT’s existing users, but would only provide insight into

previously-developed expertise. Thus, to increase the sample, it was necessary to keep the project, software and community active.

This was achieved through a series of updates to the reViSiT Pro program, addressing issues (including those exposed by the study) and adding functionality to broaden the appeal of the program, specifically to novices and new users. Each update prompted an announcement; restoring the experiment to the news cycle, increasing public exposure, and renewing interest. Appendix F details the updates, and the justification behind each. The overall success of this strategy is evidenced by the interaction spikes seen in Figure 19, each corresponding to new releases of the software. Finally, in the closing weeks of the experiment’s run, a second questionnaire was issued to gauge subjects’ subjective experience of the experiment, as well as both sequencer (e.g. host) and tracker software in general, probing factors such as their experience of flow, use of notation, and changes in their interaction preferences or perceived level of skill. The questionnaire form is presented in Appendix C, results of which are detailed in Chapter 9.

chapter six

video study: tracking composition practices

As data collection proceeded online, a video study of an expert

reViSiT composer was conducted to provide context for subsequent

analyses. This section provides an overview of the interaction in the session, and makes general observations about the captured user experience, with regard to flow, virtuosity, and liveness (see 4.2.4). In later chapters, these concepts are explored in more detail, using a much broader sample of users and scenarios. To this end, this qualitative, idiographic video case study not only provides a general overview of the tracker user experience, but formed an exploratory study that helped develop and focus later quantitative, nomothetic analyses, many of which seek to generalise findings made here.

about the task

and subject A Dutch-based user who began using reViSiT in 2006 and since

became involved in beta testing, the composer selected for the study uses the reViSiT tracker professionally, writing music for computer games, TV and film, but is also a well-known music artist from the

MSX “demoscene” (see Section 2.2.2). Outside his professional

work, for both enjoyment and practice, he also specialises in orchestral and FM synthesizer remixes and reversions of well- known electronic, film, and video game music. In this pursuit, the video records the composition of an original soundtrack for a

Warner Brothers “Road Runner” cartoon, completed over the

course of a single day (8 hours, with three 20-40 minute breaks), an intrinsically-motivated task the composer set himself. reViSiT 1.3

Pro (running in Steinberg Cubase SX3) was used and also provides

the complete log of interaction during the session (see Section 5.3 for details).

The object hierarchy shows users and sessions in the study, plus the window tree of the selected session. Selecting a window shows the map (see ) of only that window and its descendants.

2 The video pane displays and plays back recorded video footage, including audio. Videos are

synchronised with both the window simulator and session log. Scrubbing is supported using the mouse scroll wheel, which can skip by events or fixed time intervals. Below the video, the audio waveform corresponding to the before and after the current frame is displayed.

The current reViSiT focus, the editor page and (where appropriate) tab or control.

The session log displays time-stamped interaction events immediately before and after the time shown in the video. Events are colour-coded by type (e.g. keyboard, mouse, focus change), and can be shown either as a sequential list, or spaced in proportion to their timing in the log. The window simulation illustrates the changing configuration of the user’s workspace, showing window positions and the current focus (in white), within the host (red) and reViSiT (blue) software, at 1:4 scale. Mouse usage is shown using points (for clicks) and lines (for drags), lightening the corresponding part of the representation. The simulation is synchronised with the video and log, flashing the appropriate window rectangle as it receives user input. See also Figure 4.

The session overview displays an overview of the entire session log, and can be used to move within the session. The lower strip shows the distribution of events within the session, colour- coded by type (as with ). Sections with red background indicate accompanying video footage, which when active, also show a preview of the audio waveform. Right-clicking the strip opens a context menu with options controlling how the session log is displayed. Above the strip, a histogram shows the distribution of selected events within the session, based on the current event filter (see 5.4.1).

Figure 1 – Video analysis UI. Screenshot and description of the interface used to study

methodology Following an initial review of the recording, interpretation of the video and log was supported by several discussions with the composer, which are quoted as appropriate. Figure 1 describes the

Video Analysis screen of the iMPULS|IVE application (see section

5.4), showing a frame from the video recording. The camera is focused to capture the interaction around the hands and keyboard, where the majority of activity takes place.1 The mouse is only partially visible at the top of the frame, but rarely used and largely restricted to rudimentary clicks in the host. However, all window activity and mouse input is captured using log data, and simulated visually beside the video. Corresponding events in the log, as well as histograms of selected interaction events, are also displayed.

comparisons with

other studies It was hoped to conduct a similar study of sequencer use, but it

proved difficult to locate a subject to provide a useful comparison: i.e. an intrinsically-motivated composer using the software to create and edit music, in contrast to professionals (working for an external goal and reward) or studio scenarios (which focus on hardware, rather than software, interaction). Instead, references are made to another longitudinal case study of a sequencer-based composer (Collins, 2005, 2007), enabling comparisons between sequencer and tracker approaches. A screen-captured video (with inset view of the keyboard) of the Renoise tracker, made by a Renoise user presenting a tutorial, is also referenced as appropriate.2