JackTrip showed capabilities to support high quality networked audio distribution and objectively outperformed WebRTC for measured and perceived audio quality. As a result, the next tests were organised to determine network configurations that best support audio transmission using JackTrip exclusively. While the LAN-based tests presented an adequate baseline for observing the capability of real-time audio streaming over a computing network, the following set of trials examined JackTrip streaming capabilities in real-world environments independent of the Local Area Network and incorporated external computers housed outside of Anglia Ruskin University, extending distributed music applications into the Wide Area Network (WAN). The tests evaluated Jacktrip audio streaming on the commercial, commodity Internet from a residential home in London, UK to Anglia Ruskin University in Cambridge, UK. A speed test of the client computer (Figure 5.19) showed very high upload speeds, while download speeds were fairly low in comparison to the LAN network (only 4.58 Mbps), presenting a possible bottleneck for high speed audio transfer.
Figure 5.19 External client computer speed test on commercial network.
5.3.1 Investigating JackTrip Performance with Different Buffer Sizes
A number of JackTrip buffer settings were investigated, with buffer sizes of 128, 256, 512 and 1024 being compared for performance regarding dropouts, distortion, and latency. While a small buffer reduces the amount of time audio interfaces like JackTrip need to store and transmit audio packets to ultimately reduce latency, Robjohns (2008) explains that “software problems often stem from the audio interface RAM buffers being too small, and the data running out before the operating system can get back to top them up (playback) or empty them (recording).” Small buffer sizes can cause audio interfaces to run out of data to transmit before having the option to transmit it, and this can result in errors such as clicks and pops, or gaps of silence while the buffer is refreshed.
92 The first test of the wide area audio streams was conducted with a 128 size buffer and produced many audible errors, representative of loud clicks and large audio spikes in the captured recordings. These are evident as broadband noise periods in the spectrograms and can be seen in the figure below alongside the spectrogram of the source 1 kHz sine wave.
Figure 5.20 Spectrogram of example 1 kHz sine wave capture on commercial network with 128 sample buffer size (left), compared to source audio (right).
As expected, increasing the buffer size to 256 samples provided major improvements in the results as represented by the spectrogram below.
Figure 5.21 Spectrogram of example 1 kHz sine wave capture on commercial network with 256 sample buffer size (left), compared to source audio (right).
Larger buffer sizes showed successful reduction in the errors in the audio streams. Further increases of the buffer size to 512 and 1024, however, prompted warnings of latency over 30 ms from JackTrip as shown in Figure 5.22.
93 Figure 5.22 JackTrip latency warnings for delays over 30 ms.
The complete set of spectrogram results from the public network streaming tests can be found in the Appendix.
5.3.2 Dropouts at Different Buffer Sizes on Commodity Network
The number of dropouts was evaluated for the sinewave source file transmitted with different JackTrip buffer sizes. The average number of dropouts is calculated from five repeats of each test.
Table 5.6 Dropouts for sinewave tests on commercial network with different JackTrip buffer sizes. Number of Dropouts
Buffer Size T1 T2 T3 T4 T5 Average
128 12 4 23 13 18 14
256 0 0 0 0 0 0
512 0 0 0 0 0 0
1024 0 0 0 0 0 0
The initial 128 sample buffer produced a higher number of dropouts than the tests conducted over the LAN, and dropouts were consistent in each test. As shown by the spectrograms, these drops produced audio spikes and broadband noise that can make the listening experience very unpleasant. Increasing the buffer size drastically improved the results, eliminating the drops over the set of the sinewave tests.
94
5.3.3 Distortion Measurements at Different Buffer Sizes on Commodity
Network
Distortion was calculated for the sinewave recordings of the source file transmitted over the commodity Internet with different JackTrip buffer sizes. The average distortion is calculated from five repeats of each test.
Table 5.7 Distortion measurements for sinewave tests on commercial network with different JackTrip buffer sizes.
THD+N (%)
Buffer Size T1 T2 T3 T4 T5 Average
128 5.2090 1.9509 2.4970 2.5573 3.0517 3.0532 256 0.0048 0.0048 0.0048 0.0048 0.0048 0.0048 512 0.0048 0.0048 0.0048 0.0048 0.0048 0.0048 1024 0.0048 0.0048 0.0048 0.0048 0.0048 0.0048
The distortion measurements were high for the 128 sample buffer. When the buffer size was increased, the distortion was minimised and matched the source.
5.3.4 Latency Measurements at Different Buffer Sizes on Commodity
Network
Roundtrip latency measurements were taken at different buffer sizes. The average latency was calculated from five repeats of each test.
Table 5.8 Latency measurements on commercial network with different JackTrip buffer sizes. Roundtrip Latency (ms)
Buffer Size T1 T2 T3 T4 T5 Average
128 30 30 30 25 25 28
256 30 30 30 30 30 30
512 54 42 42 42 42 44.4
1024 112 112 112 112 112 112
The 128 and 256 sample buffer sizes both achieved low latency rates. Further increases in buffer sizes helped maintain streaming quality, but introduced greater latency above 30 ms.
95
5.3.5 Commercial/Commodity Network Audio Streaming Discussion
The commodity network initially displayed challenges supporting real-time audio streaming with low buffer sizes, but significantly improved when the buffer size was increased. It is possible that the lower bandwidth designations and overall network congestion on commercial networks can create more opportunities for dropped packets and errors in the data stream (Sinatra, 2014). Also, while these tests overall performed very well, the results differed from initial results seen in Hardin and Toulson (2019), showing that network congestion on different dates may severely impact performance. Additionally in addressing errors in Hardin and Toulson (2019), the use of larger buffers on commercial networks may create a scenario called bufferbloat, where latency increases due to excessive buffering of packets and ultimately reduces network throughput and successful message delivery. Nonetheless, these tests showed that in optimal conditions the current commercial/commodity computing networks have some capability of supporting real-time high quality, low latency music transfer, although comparisons to Hardin and Toulson (2019) prove that results may vary over time.