development and testing

To gather as much data as possible, from as many individuals as possible, the study has to attract people to the software and also ensure users are not deterred by other factors.

data privacy

concerns A limited number of users will join the experiment out of

enthusiasm for the research or loyalty to the developer, but most will be more circumspect about surrendering their data and privacy, especially in light of the recent blight of spyware and attempts by both hackers and companies to gather user data. Ethical approval and the mark of the University of Cambridge will allay some fears, but provide little incentive for users to update to a special version of free software that collects user data.

participation

incentives Like many experiments involving human volunteers, a more

tangible incentive is required. For small studies; sweets, vouchers or money are often used, but the size and spread of our sample makes such offerings impractical. An alternative is to make the experiment the reward itself, offering an experience that individuals would otherwise find difficult or too expensive; the promise of money saved, rather than received.

quid pro reViSiT _{Since a version of reViSiT was already freely available, there}

was little to entice current users to take part. However, prior to the research, a “Professional Edition” of the software had been mooted on the forum, promising a significantly expanded feature set, tailored and priced for professional users and enthusiasts. Consequently, it was decided to develop this edition of the program to give away to experiment volunteers. Perceived as a monetary reward, it provides an incentive to take part in the research, as well as compensation to offset privacy concerns. Figure 5 lists the four main features exclusive to the reViSiT

Professional, not only designed to extend the free edition, but

representing innovations not generally found in other programs, designed to broaden the appeal of the program to musicians working with video, audio, MIDI, and trackers, respectively.

Figure 5

exclusive features in

reViSiT Professional

surround sound support – extending stereo output to 5.1 channels, with

special notation syntax and pattern effects to address extra panning options (e.g. Cartesian or polar coords, depth and rotation slides, discrete channels).

advanced output routing – enabling different notes, samples, instruments,

or channels to be sent to the host on separate audio outputs (for effects and post-processing), where routings can be set on-the-fly using pattern effects.

MIDI-triggered patterns – allowing users to control the order of playback

via MIDI, triggering patterns using notes from a MIDI device or recorded track, enabling live arrangement or editing via the host’s visual editors.

hi-res timing & hi-def editing – addressing the perceived rigidity and low

timing resolution of trackers by enabling users to ‘zoom into’ or ‘open-up’ the pattern grid, placing musical events between rows (see Figure 1).

Once registered, a user is solely responsible for the running of the program. Whereas lab-based participants may feel morally- obliged to complete the prescribed procedure and not walkout in the middle, there is little to inhibit our user from abandoning the experiment, by simply closing and deleting the software.

reducing the impact of program errors

user feedback and testing

Program errors are especially likely to discourage and deter participants. Whereas bugs and oversights are more easily tolerated in free or academic software, users will demand a complete, fully-tested, and reliable program if it is billed as a commercial product. Moreover, errors in the program may also interfere with user interaction, where unanticipated program behaviour risks impeding the very flow sought by the experiment. As such, significant effort was invested in the testing of the reViSiT Pro, and the code to support the experiment. Figure 6 summarises the various milestones in the development and testing of reViSiT. The limited resources of lone developers are often most keenly felt in aspects of quality assurance. The user interaction and audio engine in reViSiT is complex, especially compared to more conventional effect and instrument plugins, and whilst the new features were quickly coded, debugging the complete program took significantly longer.

Defensive coding and unit testing helps reduce bugs that appear in individual lines and sections of code. Object-oriented, interface-driven coding techniques also help simplify the integration of different system components. However, it can be hard to anticipate how a system will be used, and thus many issues only present during use. The problem is arguably exacerbated for developers of authoring tools, which seek to support flexibility and creative freedom, and where original and unforeseen uses must be expected.

Figure 6

Moreover, the plugin architecture presents extra debugging challenges. The VST plugin specification (Steinberg, 1999) tries to standardise the communication and interactions between plugin and host. However, manufacturers have different approaches and styles, and thus host behaviour varies. Many hosts stray from the specification and others implement only part of it, requiring plugin developers to extensively test plugins for compatibility problems.

VST plugin

keyboard support For example, many manufacturers assume that plugins will be

manipulated using the mouse, and thus provide little or no support for keyboard input. The VST specification contains no mechanism for hosts to tell plugins what, if any, keyboard support they provide. Even where support exists, many hosts relay only a subset of keystrokes to plugins, holding back others for their own use, regardless of whether the plugin has focus or not.4

multi-threading

and thread safety Another pitfall for developers has emerged relatively recently,

and concerns the increasing use of multi-threading. As a plugin,

reViSiT exists as a dynamic-link library (dll), a set of routines that

run in the threading environment of the host program. Thread- safety problems can arise when functions are called concurrently and use shared resources (e.g. memory). Modern VST hosts run high-priority threads for audio processing in parallel with lower- priority threads for the UI, meaning users create, delete and change musical data that is, at the same time, in use by the audio engine. In the worst case, delayed execution leads to glitches or ‘dropouts’ in the sound output (for the audio thread) or stuttering and poor responsiveness in the interface (for the UI thread).

Newer versions of reViSiT use mutex objects to guard against such conflicts, where threads wait until it is safe to access an object. At the same time, identifying and debugging concurrency issues can be problematic and time consuming. Unlike some code errors, memory and resource conflicts can be hard to reproduce, and may only cause problems in a tiny fraction of use cases – when multiple threads are unfortunate enough to contend for the same object at the same moment.5

4

In tracking, keyboard support is paramount and, despite trying various workarounds (e.g. keyboard hooks and other Win32 ‘hacks’), no solution ensures uncensored keyboard input reaches the plugin window in every host. In many cases, the host’s parent window intercepts, filters, or blocks keystrokes, allowing users to click in the plugin, but not control it by keyboard. In what became an undesirable but necessary compromise, reViSiT’s UI was split into two windows: the standard VST plugin window becomes a simple, mouse-controlled transport-/tool-bar, created and positioned by the host; the main editing environment resides in a separate window spawned by the plugin itself. This window is not owned by the host, and thus receives unfiltered keyboard input from the OS directly. The impact of the split UI is minimised by making the toolbar functionality superfluous, duplicated in the editor window.

5

For example, if two threads access a shared resource roughly once a minute, and that access takes 10ms, the chance of a collision, in a given minute, could be as low as 0.000003% – the code could run continually for 8 months before an overlap occurs, which may not even then lead to an error.

automated feedback

and crash reports As such, it is not possible for developers to test all eventualities,

and the burden must be shared between a wider circle of testers, using pre-release alpha and beta versions of the software, and across different computers, hosts, and interaction styles.

Although some users have programming experience, many come from a non-technical, musical background and are unable to give technical details when issues occur; many others were more interested in simply using the program, rather than helping with testing, and don’t bother to email about problems they encounter. To address this, reViSiT 0.92.1 introduced an automated feedback system (Figure 7), allowing the program to easily send user feedback directly back to the developer, along with technical data describing the fault and code location.

Figure 7

reViSiT’s integrated

user feedback feature

(right) status bar button;

(below) feedback dialog

The mechanism can be triggered manually, to comment about the program or report a soft failure (unexpected, but non- catastrophic behaviour, not prompting an explicit error message). In the event of a hard failure (an unhandled exception or program crash), the mechanism activates automatically. In both cases, a dialog appears, asking to send technical information (a stack dump, with system and error details) and also asking the user for any additional information they can provide, including a message (error description, steps to reproduce, recent system activity, etc.), screenshot, or copy of the music data being edited.

The feature has been invaluable in fostering user feedback, and accelerating the debugging process. The stack dump can be used to recreate the crash, using a debugger, which automatically pinpoints the offending line of code and provides the function call context leading to it. In threading issues, where an error occurs in one part of the code but is triggered by another, the combination of the stack dump and a user comment reveals the nature, context and cause of the problem, which is otherwise very difficult to identify.

5.2.5. distribution

reViSiT is primarily distributed as a free download from the

developer’s website6. At times, the software has been included on magazine coverdiscs and CD shareware collections. More recently, it has appeared in major online shareware libraries, such as CNET Download.com, Tucows, Brothersoft, etc. To ensure their information remains accurate and current, a PAD (Portable

Application Description) file is now maintained on site.

Two mailing lists, covering roughly 6,000 reViSiT users and all experiment registrants, are used to announce new releases. New versions are typically released at the beginning of the week, and only announced on the forum, so that initial use is largely restricted to experienced users and enthusiasts – those most eager to trial new features and best suited to catch teething problems. In the absence of any problems, a wider release is made the following Friday to catch people as they check emails before the weekend, when they are most likely to have time to use the program. The mailing list includes several press contacts and music technology websites, who subsequently carry the announcement to their readers, attracting new users and extending the surge of registrations and downloads.