Csound on the Web

This section will discuss Csound on the Web – two versions of Csound that work cross-platform among various browsers and operating systems. The first ver- sion is built using the Emscripten [205] compiler and is a pure-Javascript ver- sion of Csound that runs in any browser supporting the WebAudio API [191]. The second implementation is a Pepper API (PPAPI)-based [80] plugin built using Google’s Portable Native Client (PNaCl) system [78] that runs cross-platform in Chrome and Chromium browsers.

The goals in extending Csound to the Web are to: • Run Csound on the client-side in a browser.

• Run Csound without the user having to additionally install anything besides the browser.

• Run existing Csound code projects without modification. • Author new Csound code content within a browser.

Being able to reuse Csound within the browser offers existing Csound users a way to apply their existing knowledge to create web-based projects, without having to learn or create a new computer music system. Conversely, new users who might learn to use Csound in a browser have the opportunity to transfer those skills to create desktop and mobile-based music applications. Finally, by creating web pages that can render Csound code without requiring installation of any plugins or applications, the web offers a solution for both long-term preservation of works as well as ubiquitous sound and music computing.

The following will begin with setting out common design goals for both Web versions of Csound. Next, the implementation of Emscripten and PNaCl

ports will be covered separately. Finally, a comparison of the two ports will be provided.

Design

When Lazzarini, Costello, and I first started looking at Csound on the Web, our technical goals were to:

• Have a complete version of Csound available for building web applica- tions.

• Build the web version of Csound using the same source code as the desktop and mobile versions of Csound.

• Create real-time and non-real-time music applications written using HTML, Javascript, and Csound.

• Provide simple deployment of applications and pieces.

• Have applications run client-side within the browser, using web tech- nologies that do not require any installation of plugins.

• Be cross-platform.

It was important to the Csound developers that we look at solutions that did not require rewriting Csound in another language (i.e., Javascript), but that instead used the same C source code as the desktop and mobile versions of Csound. Using a common codebase was critical, as it would have been time consuming for the community to maintain and test a separate version of Csound. A common codebase also ensured that the same Csound music code would render in exactly the same way regardless of what platform it is on, including the Web.

We also wanted the development experience of web applications using Csound on the Web to mimic very closely the experience of developing applications on the desktop and mobile platforms. The expectations are that the same Csound music code could be used in any project, and the only code that would need porting would be specific to each target platform. To achieve that end, we wanted to provide a Csound API implementation that was very similar to those provided on other platforms. That way, users who start by building a web application could more easily port their code to a mobile or desktop platform, and vice versa.

Furthermore, we wanted to ensure that Csound-based web applications were cross-platform across operating systems. We also wanted the implemen- tation of Csound on the Web to be cross-browser, though this was less of a priority than working across operating systems. Working across operating systems meant that a user could create a project and have it run on as many computers as possible. Working across browsers would add an additional level of reach so that users could open projects in their own preferred browser.

After evaluating the available client-side web technologies, two systems stood out: Emscripten and PNaCl. These two systems provide viable solutions for the goals we wanted to achieve. However, the two systems also come with certain tradeoffs. The following will discuss to the two technologies and the approaches we took in employing them. Afterwards, I will provide an analysis of the tradeoffs between the two systems.

Emscripten

Emscripten [65] is an LLVM-based compiler technology that allows cross- compilation from C and C++ into a subset form of Javascript called ASM.js [88].

As ASM.js is a subset of Javascript, code produced by Emscripten runs in any Javascript interpreter. However, certain browsers, such as Firefox, are optimised for ASM.js and are capable of running ASM.js code faster than standard Javascript code. Emscripten has been used to port games and other applications to run within the browser.

As the result of running Emscripten is ultimately Javascript, the limitations of using this technology are mostly a result of the services and technologies provided by the browser itself. For Javascript, the current standard to generate and process audio is done using the WebAudio API. (The WebAudio API, in its current form, has limitations that influence the usability of Csound. These limitations will be discussed below.) Also, one of the biggest hurdles for Emscripten use at this time is that it does not support threads – in particular, the cross-platform pthreads library.

Getting Csound to operate in the browser via Emscripten was largely done in two parts. Firstly, we wanted to get libsndfile and the standard libcsound C API compiled and available as a Javascript library. Secondly, we needed to wrap the compiled library to connect it to the audio system of the browser via the WebAudio API. The first part was done primarily by myself and required modifying Csound’s source files and CMake build files to compile Csound with Emscripten’s toolchain. The latter part was done primarily by Edward Costello to design and implement the WebAudio connection with Csound, as well as build up the platform-specific CsoundObj-like API for this project.

To get Csound compiled with Emscripten, we first had to build libsndfile. Using libsndfile’s own autoconf-based build, together with Emscripten’s build toolchain, we found we only required one patch [201] to the source code to compile libsndfile into an LLVM bytecode (.bc) library. The generated target

at this stage is LLVM bytecode as this is later used for linking, before the final Javascript code is generated.

Compiling Csound with Emscripten was a larger challenge due to Csound’s dependence on the pthreads library. Originally, a platform-specific, Csound- provided threads implementation [10] was used throughout the codebase. Later, the platform-specific parts were removed and only two implementations remained: one implemented with pthreads and the other providing a dummy implementation. However, as the pthreads library was set as a required library in the build system, the dummy implemented was no longer used. Also, as pthreads was assumed to always be available, we found that pthreads-provided functions were being used directly in the codebase.

To get around the issue of pthreads, both the build system and the source code were changed. Firstly, the CMake build system was modified to ignore searching for pthreads if and only if building with Emscripten. This change was limited to the Emscripten-build to ensure that if pthreads was not found on other platforms where it should be available, then it would cause a build- time failure. After the changes in CMake, code within the Csound codebase was modified with pre-processor conditional checks to optionally compile certain code when building with Emscripten.

The result was that libcsound could be compiled without thread support. This allowed for single-threaded use of Csound, with all of the same built-in opcodes and features as is found on all other libcsound builds (i.e., Desktop, Mobile). For the purpose of using Csound with WebAudio, being single- threaded was sufficient.

After libcsound and libsndfile were compiled, Costello then wrote an implementation of CsoundObj in Javascript. This version of CsoundObj, like

the Android and iOS versions, comes pre-configured to handle Csound and platform-specific I/O to work with the native audio system (i.e., WebAudio). It does this by creating a WebAudio ScriptProcessorNode [125] that manages and runs an instance of a Csound engine. The ScriptProcessorNode is responsible for transferring incoming samples to Csound, running the Csound engine for n number of buffers, and transferring samples back from Csound to WebAudio.

The current version of Emscripten CsoundObj does not provide the CsoundObj Binding system found in Android and iOS.15 _{Instead, users read}

values from and write values to Csound using its channel system. Whether Bindings will be implemented is currently on hold as the future of WebAudio is moving away from the ScriptProcessorNode and towards AudioWorkers. With the the current state of AudioWorker design, AudioWorkers will run in a separate thread and can not share memory with the main JS thread. This would make it impossible to implement a synchronous Bindings system. It is unknown whether Bindings can be implemented for Emscripten Csound until the final design of AudioWorkers is complete.

Like iOS and Android, a Csound Emscripten SDK is now released together with every new version of Csound. The SDK provides a pre-compiled version of libcsound.js (generated from libcsound and libsndfile), CsoundObj.js, and a set of HTML-based examples that demonstrate usage of Emscripten Csound and also act as a manual for the project.

PNaCl

Another technology, Google’s Pepper API (PPAPI) and Portable Native Client (PNaCl), offers a different path to developing cross-platform, web- based audio applications. With PNaCl, C and C++ code can be compiled into a portable bytecode that is in turn ahead-of-time compiled just before run-time. Developers compile applications into portable .pexe files, deploy them over the web together with a web page, and run them across browsers that support PNaCl. Additionally, web pages and PNaCl applications can communicate with each other using the PPAPI. This API provides a standardised way for C code to receive messages and data from Javascript, and, in turn, send messages back to Javascript.

Google provides a cross-platform compiler toolchain for compiling C and C++ code into PNaCl binaries. As Google creates both PNaCl and Android, it is unremarkable that the PNaCl toolchain is very similar to the one provided for Android’s NDK. As a result, the build for PNaCl Csound is setup similarly to the one used for Android Csound, with a custom Makefile used instead of an Application.mk and Android.mk file.

Because PNaCl supports pthreads and the toolchain is very similar to ones provided for platforms like Linux, there were few changes necessary to build libcsound and libsndfile. After doing the initial build, Lazzarini then filled in the parts normally implemented within a CsoundObj implementation though, in this case, it is a Javascript and C++ class called Csound.16 _This

Csound C++ class uses the Pepper API to register callbacks and handle audio

16_{The PNaCl Csound class could easily be renamed to CsoundObj to provide closer}

parity to CsoundObj implementations on other platforms, as it functions in the same ways, wrapping portable libcsound API usage with platform-specific code for PPAPI.

communication between a Csound engine instance and the MediaTrack audio system provided by Pepper. The C++ class also handles receiving messages to perform actions such as starting, running, and stopping a Csound engine instance. The Javascript class in turn uses the Csound C++ class via the PPAPI and wraps the functionality of the C++ class, so that end users can write their web applications using Javascript to work with Csound.

Csound PNaCl releases are now released together with every new version of Csound. The SDK follows the norms of other Csound-based SDKs and provides a pre-compiled library, example code, and a manual.

Comparing Emscripten and PNaCl

While both Emscripten and PNaCl versions of Csound provide the same libcsound-based features and are both capable of compiling and running the same Csound code, the two implementations differ in significant ways in terms of performance and availability. In terms of performance, as shown in [108], the Emscripten build does not run as fast as the PNaCl one in terms of raw speed. Additionally, and perhaps more importantly, the WebAudio ScriptProcessorNode’s design is not optimal for real-time audio. Consequently, breakups in audio are much more likely to occur with the Emscripten build than in the PNaCl build. For PNaCl, the audio system interaction between Csound and Pepper API’s audio system is very similar in design to how Csound interacts with desktop audio systems. As a result, for the best real-time audio performance, the PNaCl version is much preferred to the Emscripten version.

However, in terms of availability, while PNaCl and the Pepper API is a cross-platform technology, it is only implemented by a single browser vendor

– Google – in their Chrome, Chromium, and ChromeOS browsers. On the other hand, since the Emscripten build of Csound only requires Javascript and WebAudio, it is capable of running everywhere that the PNaCl build is able to be run (i.e., the previously mentioned Chrome browsers), as well as all of the other browsers that provide WebAudio (i.e., Firefox, Safari). For users who are less concerned with performance and more concerned with availability, the Emscipten version would be much preferred over the PNaCl version.

With either version of Csound for the Web, the end-user does not require separately installing any applications or plugins prior to loading the web page. The user can just load the page and everything will download and run. From the perspective of dependency management, one can view the Web Csound builds as having all of their library dependencies statically compiled into their builds, such that they will always be satisfied when running the web page. The result is that existing Csound-based projects can be preserved using the Web, together with the exact version of Csound used to create the project. Also, new Csound-based projects will have a high degree of long-term viability to execute and run in the future.