Prototype Build and Test

Facilitating the practical nature of this research, the main approach for the design, build, and test stages follow a waterfall-style methodology. The waterfall method detailed by Keith (2010) is a commonly used methodology in product development and follows a list of discreet steps or phases that begin with an initial concept and lead to the testing and evaluation of a finished product. Furthermore, each subsequent phase builds upon the results of the previous stage to accomplish a larger and more complex overall goal.

Figure 3.4 Example of a Waterfall game development method.

The overall design and build occurs in stages; once the initial concept is specified, small targets and testing milestones are set, eventually culminating into a final design. Adding an agile design component, the design and build stage incorporates an “inspect and adapt” cycle that analyses the strengths and weaknesses of each developed stage and informs decisions to adapt to new goals when significant changes need to be made (Keith, 2010, p. 30).

34 The major stages in the design and build process is laid out below:

Stage 1: Embedded systems and fundamental IoT processes research

The initial research stage (see Figure 3.1) involves exploring emerging developments in the Internet of Things and examining the leading technology to determine which hardware and/or software solutions can sufficiently support remote interconnectivity and control in audio applications. These solutions are based upon low cost computing resources and publically available open source code that are readily available to the general population and do not require commitments to proprietary commercial products.

The key technologies utilised in this research are: Microcontrollers

The primary microcontroller used for this research is the Freescale FRDM-K64F. It uses a 32-bit core ARM® Cortex™ M4 processor with 1024 KB of flash memory and 256 KB of RAM (ARMmbed, n.d.a). Added benefits to this particular microcontroller is that it comes equipped with an Ethernet jack for physical network connections and provides extension sockets allowing easy connections for external sensor modules and shields that plug directly into the board. This allows prototyping technical designs and concepts to be simpler and more efficient.

HTML5 Websockets

The Websockets protocol allows full-duplex, bi-directional communication between a computing server and client over the Internet. HTML5 Websockets is an application programming interface (API) that is implemented into a web browser enabling the browser to set up direct communication with a server in real-time, and this communication can also extend to other clients (i.e. a networked microcontroller) connected to the server (Kaazing, n.d.). Consequently, HTML5 Websockets set the frame work for establishing the graphical interface to interact with the connected IoT devices and allows the design to be more portable across mobile devices.

Webservers

A webserver takes online requests from a client (ex: a computer user trying to access a webpage), processes the requests, and serves the client with the desired information. Specifically for this research, the webserver mediates the communication between the embedded IoT devices and other networked

35 functions. The webserver used in this research is the Tornado webserver and is programmed in the Python programming language.

JackTrip

JackTrip is a Mac OS and Linux based system for streaming high quality, uncompressed audio over the Internet. It can support a max amount of audio channels governed by the computers capacity, and can manage bi-directional audio streams (Caceres, 2007a).

WebRTC

WebRTC is an open source platform that allows real-time communication of audio, video, and data over web browsers and mobile devices. WebRTC incorporates APIs such as getusermedia allowing access to a computer’s internal and external media devices to transfer media data over the web. While WebRTC contains its own APIs for capturing media, it relies on other resources such as signalling and NAT and firewall traversal to share data between peer computers.

Motors/Actuators

The use of motors serves as the bridge between the digital and physical components. Motorised potentiometers in particular allow voltages, commands, and input values from digital sources to be interpreted into physical movements that can control the rotation of a knob or the sliding of a fader while still allowing physical, hands-on interactions to achieve the same effect. Motors connected to microcontrollers allow virtual interactivity with physical components.

Stage 2: Design and prototyping IoT control systems

Two microcontrollers were chosen for this research: the mbed LPC1768 and the FRDM K64F. The mbed LPC1768 was one of the original development boards designed by ARM and offers a great deal of developer support for application prototyping. The FRDM K64F is designed by Freescale, but runs ARM’s M4 Cortex processor and offers 1024 KB of flash memory (twice that of the LPC1768) and 256 KB of RAM (8 times that of the LPC1768), making it powerful and capable of managing a range of immediate computing requests (ARMmbed, n.d.a). For transmission of control information, HTML5 Websockets stood out as a robust communication platform to support a virtual control infrastructure between a user interface and microcontroller processing unit. Websockets allow for bi-

36 directional communication over the Internet between a server and a client, and can support a HTML5 web-based graphical user interface.

The current webserver is programmed in the Python programming language and incorporates Websockets headers to allow transmission of commands between a server and client. During this stage the Websockets server and client HTML5 webpage were created and configured to establish an interface for Internet-based control of the microcontrollers and the physical hardware systems. Upon successful configuration of the microcontrollers with the client interface, actuators were incorporated to demonstrate the capabilities of fully developed, IoT control system that can manipulate controls on a physical audio device.

Stage 3: Investigating, testing, and incorporating audio streaming functionality Modern audio streaming platforms were evaluated for their effectiveness in audio transmission to support an IoT-enabled music application. The JackTrip application was eventually selected as the primary audio streaming application in this research after conducting local and wide area streaming tests to evaluate the quality of network-based audio transmission and robustness in managing network data loads. Application testing showed that JackTrip accurately maintains the qualities and properties of source audio files when streaming over the Internet, making it the optimal for a networked music application. Initial tests delivered several instances of audio dropouts and errors in the streams, however, further evaluation discussed later in this research determined that JackTrip performs better over specific types of networks (i.e. local area and high speed educational networks) as compared to traditional, public networks. JackTrip was compared to WebRTC, another real-time audio streaming platform used primarily for Skype-like voice and video applications within web browsers, and ultimately outperformed WebRTC in evaluations conducted in this research. Details of the audio streaming evaluations are presented in Chapter 5.

Stage 4: Design and prototyping hybrid analogue-digital IoT audio systems

The last design stage combines elements of the IoT control systems and audio streaming phases to create a final IoT enabled audio system. Main focuses involve centralising control, audio, and an additional visual element into an integrated user experience. The prototyped IoT audio system focused on two audio processors: a physical, analogue music device that would be embedded with computing hardware creating a hybrid,

37 digitally-controlled analogue device and the second being a unique space whose acoustic qualities could be used for a real-time reverb application. This is discussed in further detail in Chapter 4.