An Evaluation of the Potential of Synchronized
Time to Improve Voice Over IP Quality
Hugh Melvin and Liam Murphy
Abstract— Delivering PSTN-like quality over current
best-effort Internet infrastructure presents many technical challenges. Much research in recent years has focused on receiver-based ap-proaches which adapt to varying network conditions in order to optimize playout quality. In this paper, we propose and evaluate a receiver-based approach that implements a hybrid adaptive-fixed playout regime by integrating synchronized time into the playout algorithm. Such an approach can deliver significantly better quality than existing adaptive techniques particularly when the underlying network is not heavily congested and end-to-end delays are not excessive. We present some initial results from our testbed system using the ITU-T E-model to quantify improvements.
I. INTRODUCTION
T
HE degree to which Internet Telephony has replaced the traditional PSTN has to date been very limited [1]. The Internet’s best-effort service compares poorly with the deterministic, circuit switched PSTN network. The ITU-T recommendation G.114 specifies that one-way delays should not exceed 150 msec [2]. With the exception of satellite links, the PSTN easily meets these limits, presenting end-to-end delays of which propagation time forms a significant and consistent proportion. The Internet generally presents higher and more variable end-to-end delays where propagation time is often dwarfed by congestion delays. In addition, severe Internet congestion often leads to packet loss. The ITU-T E-model, though limited enables such delay and loss to be quantified on an additive Transmission Rating factor (R) 0-100 scale, which can be mapped to the better known Mean Opinion Score (MOS) 1-5 scale [3]. It is significant that R (& MOS) values are more sensitive to an increase in packet loss than to an increase in delay [4].Because of the commercial worth of voice services, signifi-cant research has been undertaken in attempts to improve VoIP quality of service (QoS). Such research can be categorized by whether it involves measures taken at the sender, in the network, or at the receiver [5]. In this paper we focus on receiver-based buffering approaches. These compensate for network jitter by delaying playout to facilitate the arrival of delayed packets, at the expense of adding to the overall end-to-end delay. We summarize such work outlining both the merits and shortfalls of the differing approaches. We also propose, describe and evaluate a new hybrid adaptive-fixed playout strategy. In this approach, synchronized time, provided via the
H. Melvin is a Ph.D. student at the Department of Computer Science, University College Dublin, Ireland (email: hugh.melvin@nuigalway.ie). Liam Murphy is a lecturer at the Department of Computer Science, UCD and the director of UCD’s Performance Engineering Laboratory.
Network Time Protocol (NTP) is used to determine end-to-end delays on a per-packet basis and this precise information is used to select an optimum playout strategy. Results were gathered from a testbed that linked Dublin City University with the National University of Ireland, Galway. Extensive tests were carried out over a number of days under varying network loads. Our results indicate an R-scale improvement of up to 25 when compared to baseline existing adaptive approaches. More generally we propose that a hybrid adaptive-fixed strategy can significantly improve quality on Internet links where the network is not heavily congested, and actual end-to-end delays rarely exceed ITU-T G.114 requirements.
The remainder of the paper is organized as follows. Section II outlines and evaluates some alternative receiver-based buffer strategies, introduces the issues surrounding accurate delay measurement, and describes the hybrid algorithm developed by the authors. Section III outlines the use and limitations of the ITU-T E-model. Section IV describes the test system in detail. Section V presents detailed results including an analysis of NTP performance. Section VI concludes the paper, outlining further work currently being implemented.
II. RECEIVER BUFFER STRATEGIES
Jiang and Schulzrinne outline that the total end-to-end delay in a VoIP session includes [6]:
Operating System delay.
Host hardware input/output delay including packetization, encoding and decoding delays.
Network delay comprising transmission delay, propaga-tion delay and variable queuing delay within intermediate routers and switches.
Application delay, introduced in the receiver to absorb the variation in end-to-end delay principally due to queuing delays.
From a packet loss perspective, it is important to distinguish between link loss and late loss. The design of the Application delay involves a trade-off between total end-to-end delay and late packet loss.
Different approaches have been taken in the implementation of application delay or buffering within the receiver. Refer-ence [7] compared four playout delay adjustment algorithms, two of which (denoted Alg. 1 and 4 in [7] and known as Alg. A and B respectively in this paper) are based on stochastic gradient algorithms. They differ in that Alg. B additionally employs a spike detection mode. In non-spike mode, both algorithms use a linear filter mechanism that tracks network conditions (see 1) and adjusts playout time accordingly at the
start of a talkspurt as given by (2). (1) !#"$% & (2)
In the above, ' refers to packet ',
is the estimated end-to-end delay,
is the filter gain,
is the measured delay,
is the playout time,
is the send time, &
is the estimated variation in delay and
"
is a multiplication factor. Adjusting on a per-talkspurt basis maintains the integrity of speech within talkspurts whilst altering the inter-talkspurt silence periods.
Moon, Kurose and Towsley propose a different approach in [8] (denoted Alg. C in this paper) whereby delay percentile information for 10000 packets is maintained and updated with each received packet and used to dynamically adjust playout delay, again on a per-talkspurt basis. They also incorporate a spike detection mode and report significant benefits from their approach when compared with Alg. A and B. They conclude that:
Alg. A reacts too slowly to sudden network variations and thus is only suitable for slowly changing network conditions.
Alg. B tends to underestimate playout delay particularly after exiting spike mode due to its gain values.
More recent work by Liang, Farber and Girod [9], de-noted Alg. D in this paper, propose yet another mechanism that differs significantly from the above. Similar to Alg. C, it maintains a continuously updated histogram of previous packet delays to predict future playout delay. In contrast to Algorithms A, B and C, the playout adjustment of Alg. D is made on a per-packet basis. This is achieved by compressing or elongating packets (known as scaling) using a technique based on the Waveform Similarity Overlap-Add (WSOLA) algorithm. They report from tests that such scaling of packets results in little degradation of audio quality (0.3-0.5 in the DCR (Degradation Category Rating [10])) though qualify this by noting that scaling occurs infrequently during the reported tests. They note also that per-talkspurt algorithms will fail to react to short spikes where such spikes are contained within a single talkspurt. Considering the composition of human speech [11], this limitation can be very significant.
Finally in [4], further comparisons are made between fixed and per-talkspurt adaptive playout algorithms (Alg. B above). The main points of this analysis are as follows:
Adaptive algorithms that utilize the TCP-like formula such as (1) and (2) tend to overestimate delays.
The selection of parameters such as
and
"
is a non-trivial matter and tuning is required to suit network characteristics. Such characteristics may however change with time.
Note that Alg. A-D are designed to operate with some late loss although the degree to which they do so varies. Additionally Alg. A-C suffer from distortion of inter-talkspurt silence periods and Alg. D suffers from packet scaling (distortion of both silence periods and talkspurts). Adaptive algorithms are thus most useful when receivers have no knowledge of actual one-way delays which is usually the case. If actual delays are
known, and are within G.114 requirements, a fixed playout delay can avoid many of the problems associated with adaptive algorithms. However a fixed playout delay, imposed at the start of a session may quickly become inappropriate resulting in high late losses or unnecessarily high delays.
A. DELAY MEASUREMENT
Accurate delay measurement within the Internet is a non-trivial issue. Montgomery evaluated a number of solutions in [18], including the use of Round Trip Times (RTT), dis-tributed synchronized time and a novel approach involving a variable delay estimation mechanism within routers, requiring a specific protocol format. More recently, RFC 2679 [19] examines the core issues surrounding accurate delay measure-ment. RTT remains an unreliable delay estimation mechanism but the availability of distributed synchronized time has greatly increased in recent years. This is due in part to the widespread deployment of the Network Time Protocol (NTP) but more importantly, to the availability of cheap yet highly accurate time sources such as GPS receivers.
B. HYBRID ADAPTIVE-FIXED ALGORITHM
The authors have implemented a hybrid adaptive-fixed algo-rithm that combines the useful characteristics of both adaptive and fixed buffer strategies. It operates as follows:
Session commences implementing an adaptive buffer algorithm.
The availability of synchronized time enables one-way packet delays to be precisely determined. Each receiver builds up a histogram of delays and from this extracts a delay estimate value(*)
to meet target loss requirements. In the following pseudocode,+-,
refers to the packetiza-tion delay and.0/ refers to a weight factor (0-1) applied
in determining the fixed playout delay. Recall that the G.114 limit is 150 msec.
If (est < (150-Pkt)) {
playout = est + (150-Pkt-est) * Wf
Fixed Playout Mode = TRUE // switch to fixed mode }
Else
Maintain adaptive playout mode
Maintain rolling histogram and at periodic intervals, recalculate delay estimate (*)
:
If (est < (150-Pkt))
playout = est + (150-Pkt-est) * Wf If (Fixed Playout Mode)
If (conditions improved)
Decrease fixed playout delay Elseif (conditions deteriorated)
Increase fixed playout delay Else
Fixed Playout Mode = TRUE // switch to fixed mode
Else // high delays If (Fixed Playout Mode)
Fixed Playout Mode = FALSE // switch to adaptive mode Else
Maintain adaptive playout mode
The hybrid algorithm maintains a profile of network delays before determining a fixed playout delay, and thus avoids the
inflexibility usually associated with fixed buffer algorithms. The weight factor .0/ provides an extra delay margin by
positioning the fixed playout delay between the extracted delay estimate (*)
and the G.114 limit. Increasing .0/ will result
in greater reductions in late losses at the expense of higher fixed playout delay. By implementing a fixed playout whenever possible, the integrity of speech both within and between talkspurts is maintained. Adaptive mode is thus used only when absolutely necessary.
III. ITU E-MODEL
The E-model is an ITU-T standardized tool for predicting how the average user rates the voice quality of a phone call with known transmission parameters. Technical details are specified in [3]. The model returns a transmission rating factor R, defined as:
0
The factors of interest are
(delay impairment) and (loss impairment). includes the distortion caused by low bit rate codec operation as well as the effect of packet loss (both link and late loss). Both
and
are intrinsic to the voice signal whereas factor
is an advantage factor. All three are not considered in this analysis. In the context of evaluating real voice sessions, the E-model is limited in that it delivers an instantaneous rating based on singular loss and delay figures. Other research has thus examined issues such as bursty versus random loss, recency [12] [4], perceived versus instantaneous quality [12] and rating of entire voice calls rather than segments [4]. For the purposes of this paper, we use a simplified yet conservative E-model analysis to compare the performance of the hybrid algorithm with per-talkspurt adaptive algorithms. As such, our analysis is aimed at extracting approximate relative values rather than detailed absolute values.
In considering
, we simplify the effect of both talker and listener echo by applying a 51dB loss value which corresponds to reasonable echo cancellation. As such, the
impairment can be approximated by 10 units per 100 msec [4] [13]. Note that acoustic echo is a serious problem with PC-based speaker/mic kits [13] leading to much worse impairment values than those assumed here.
Regarding the factor, all tests utilized the G.711 codec which introduces almost no distortion, compared to lower bitrate codecs such as GSM or G.723. For the purposes of quantifying
per percentage packet loss, we presume firstly that PLC (Packet Loss Concealment) is applied thus reducing the impact of packet loss [see G.113 [14] for details]. Secondly we determine the impairment value on the basis of bursty rather than random loss. From [14],
for packet loss in the range 0-5% can therefore be approximated by a dual-slope curve set at 10 units per % above 3% and at 3 units per % below 3% (packet loss rarely exceeded 5% during the tests). Note also that values from [14] are based on 10 msec G.711 packets rather than the 30 msec packets used in our tests. As such we underestimate the impairment as [16] illustrates for G.723. A critical factor for the hybrid algorithm
GPS Satellite DCU NUI,G NTP Server, NUI,G NTP VoIP NTP VoIP GPS Receiver GPS Satellite Internet HEANET Testbed Implementation NTP Local Client NTP Remote Client NTP NTP VoIP NUI,G LAN Internet Remote NTP Servers NTP
Fig. 1. Testbed system
is the lower sensitivity of
to increased delay than that of
to packet loss. Practically speaking, this means that users are more tolerant of higher delay than higher loss. This justifies the use of the weight factor .0/ in calculating the fixed playout
delay from the delay estimate (*)
. A simple mapping exists from values to the 1-5 MOS scale [3].
IV. IMPLEMENTATION DETAILS
Fig. 1 describes the test system developed for evaluating the hybrid algorithm. The end-hosts, one at the National University of Ireland, Galway (NUI,G) and the other at Dublin City University (DCU) are 220km apart and connected by the Higher Education Authority network (www.heanet.ie). HEAnet is Ireland’s academic and research network and is linked to similar networks in Europe and the US. HEAnet is a well-provisioned network and considering its limited geographical size, internal network delays are usually well within the G.114 limit.
Synchronized time was implemented via NTP. A stratum 1 NTP server was set up at NUI,G to strengthen the local NTP infrastructure. A GPS clock provides its reference source. Six other stratum 1 NTP servers, located in the UK, France, Germany and Switzerland were included in the NTP client configuration files. For more detail on the implementation of NTP within the test-system see [5]; for greater detail on NTP operation see [17].
An open source implementation of H.323 (www.openh323.org) was used to deliver pre-recorded voice streams. This code was modified to extract trace data for simulation and analysis, and indeed more fundamentally, to implement both adaptive and hybrid algorithms in realtime. A Matlab based simulator was also used to verify the correct operation of the modified code.
The adaptive algorithms chosen for comparison were Alg. A (with =0.98 and " =4) and Alg. B ( varied from 0.875 to 0.98,"
set to 4). The weight factor.0/ used within the hybrid
algorithm was set to 0.33, based on network characteristics. Its value represents a trade-off between total end-to-end delay and the acceptable degree of late loss and is influenced by network jitter.
Each end-host advertises its implementation of NTP. This is verified by sending an
20 40 60 80 100 120 140 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2
Sample Number (taken every 15 min)
Offset (msec)
Offset of NUIG client to candidate servers
NUIG Server ntp−sop.inria.fr ntp1−rz.rzze.uni−erlangen.de ntp2.ja.net hora.cs.tu−berlin.de swisstime.ee.ethz.ch ntp2.ptb.de 23:00 hrs 04:00 hrs 09:00 hrs 14:00 hrs 19:00 hrs 24:00 hrs
Fig. 2. Performance of the NUI,G NTP client
analyzing the response. As detailed in [17], a robust NTP subnet requires careful consideration and design.
The integration of synchronized time within the hybrid algorithm is done through the Real-Time Transport Protocol Control Protocol (RTCP) Sender Report (SR) packets. RTCP is a companion protocol to RTP and packets are generated periodically during a session [20]. On receipt of the 1st RTCP SR packet, each receiver extracts the NTP and RTP timestamps contained within the packet. As system clocks are synchronized through NTP, the receiver can therefore determine all subsequent incoming RTP packet transit times.
V. RESULTS
Before evaluating the relative performance of the algo-rithms, it is important to assess the performance of NTP as synchronized time is a critical requirement for the hybrid algorithm. Fig. 2 & 3 outline the performance of the NUI,G and DCU client NTP hosts respectively over a 36 hr period, during which the tests were carried out. The legend outlines which of the seven servers were candidates at each sample instant plus the clock offset of each of the candidates relative to the client clock. The NUI,G client clock offset remained within a +/- 1 msec band for 97% of the time whereas the DCU client clock offset remained within a +/- 7.5 msec band for 95% of the time. As expected, the NUI,G client remained tightly coupled to the NUI,G NTP server, selecting it as a candidate 93% of the time. The DCU client selected the NUI,G client as a candidate 60% of the time, disregarding it whenever the DCU-NUIG link became congested, relative to the other servers. This is reflected in the range of offset values and confirms the need for robust local NTP infrastructure. Overall within a VoIP context, the performance of both the NUI,G and DCU NTP clients was very satisfactory. In all 40 tests were carried out over a 3 day period. Figures 4, 5 and 6 summarize results for one of the 3 days whereas Fig. 7 outlines a single test result during a period of significant congestion. The latter compares the performance of the hybrid algorithm with that of Alg. A. In all but 3 of the tests, the hybrid switched from adaptive to fixed at the 1st opportunity (after 1st RTCP Sender Report is received) and remained in fixed playout mode. As illustrated in Figures 4 and 5, the hybrid algorithm resulted in a late loss reduction of up to 4 percent whilst incurring an increase in end-to-end delay of 30-40 msec. Note that Alg. B, though reacting well to spikes, performed poorly, resulting
20 40 60 80 100 120 140 −15 −10 −5 0 5 10 15
Sample Number (taken every 15 min)
Offset (msec)
Offset of DCU client to candidate servers
NUIG Server ntp−sop.inria.fr ntp1−rz.rzze.uni−erlangen.de ntp2.ja.net hora.cs.tu−berlin.de swisstime.ee.ethz.ch ntp2.ptb.de 23:00 hrs 04:00 hrs 09:00 hrs 14:00 hrs 19:00 hrs 24:00 hrs
Fig. 3. Performance of the DCU NTP client
10:000 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 1 2 3 4 5 6 7 8 Time of test
Late loss rate %
Late loss rate for different algorithms
Algorithm A Algorithm B Hybrid
Fig. 4. Late Loss Reduction due to hybrid algorithm
in high late losses when exiting spike mode. As outlined in section III, a simplified E-model analysis was applied which focuses on the relative performance of the algorithms rather than absolute R or MOS scores. Applying the approximate
and
impairment formulas from section III to Figures 4 and 5, Fig. 6 shows that the net effect of the hybrid algorithm was an R-factor improvement of between 2-25.
Although Fig. 7 shows a high variance in network delays over a specific test period, the long term variation in delay over the testbed link was much less dramatic. Fig 8 shows the delay seen by the DCU NTP client host whilst querying the NUI,G server over a 36 hr period, during which the tests were carried out. Congestion was not a serious problem on this link and other than very infrequent spikes, the delay remained around 16 msec. In fact, analysis of the delays to the other six NTP servers, none of which is located in Ireland, indicates that the delay pattern was quite similar although the baseline delay ranged from 30-60 msec depending on location. In [21], the
10:000 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 5 10 15 20 25 30 35 40 45 50 Time of test
Delay Penalty (msec)
Additional delay due to hybrid algorithm
Hybrid delay penalty relative to Alg. A Hybrid delay penalty relative to Alg. B
10:00 0 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 5 10 15 20 25 30 35 40 Time of test R−factor improvement
R−factor performance comparison
R−factor improvement : Hybrid v Alg. A R−factor improvement : Hybrid v Alg. B
Fig. 6. R-factor improvement due to the hybrid algorithm
0 500 1000 1500 2000 2500 3000 3500 0 20 40 60 80 100 120 140
Performance of Hybrid vs Adaptive algorithms
Packet Number
Delay (msec)
Hybrid changeover to fixed playout
Subsequent c/o to higher fixed playout
Adaptive playout Hybrid commences in adaptive mode
1st RTCP packet received => analysis of actual delays commences
Fig. 7. Typical test result
authors extend the evaluation of the hybrid to more diverse networks and network conditions. Results are consistently encouraging, indicating that the hybrid algorithm is a viable alternative over much of the Internet’s infrastructure.
VI. CONCLUSIONS
In this paper we propose and evaluate a new hybrid adaptive-fixed playout algorithm for VoIP. We describe and evaluate existing adaptive algorithms. Developments in NTP and GPS technologies have greatly facilitated the implemen-tation of distributed synchronized time and thus accurate delay measurement. Where network delays are known, and are within G.114 requirements, adaptive playout is largely unnecessary, and results in unnecessary late packet loss and voice distortion. In such cases, a fixed playout delay, though increasing overall end-to-end delay, will significantly reduce
20 40 60 80 100 120 140 0 10 20 30 40 50 60
Delay seen by DCU to NUI,G
Sample Number (taken every 15 min)
Delay (msec)
23:00 hrs 04:00 hrs 09:00 hrs 14:00 hrs 19:00 hrs 24:00 hrs
Fig. 8. Delay to NUI,G NTP server seen from DCU
late packet loss and fully preserve speech integrity. This trade-off can often result in improved quality as users are more sensitive to increased loss than delay. Our hybrid algorithm combines the useful characteristics of both adaptive and fixed buffer schemes, estimating and implementing an optimum fixed playout delay whenever possible whilst utilizing an adaptive playout scheme only when necessary. Our results indicate an R-factor improvement of up to 25 over a series of tests carried out within the Irish research network. Finally, we show that NTP, though requiring careful subnet design, more than adequately supports the implementation of the hybrid algorithm.
Current work is examining the benefit of dynamically vary-ing, both the weight factor and the frequency of
playout-mode re-evaluation, according to network conditions. Due to the E-model’s limitations, more analysis including subjective testing is also required to accurately quantify relative perfor-mance gains.
REFERENCES
[1] S. Bradner, “Internet Telephony-progress along the road,” IEEE Internet
Computing, vol. 6,no. 3,May-June 2002, pp37-38.
[2] Recommendation G.114, “One way transmission time,” ITU, May 2000. [3] Recommendation G.107, “The E-model, a computational model for use
in transmission planning,” ITU, May 2000.
[4] A. Markopoulou, F. Tobagi, and M. Karam, “Assessment of VoIP quality over Internet backbones,” IEEE Proc of Infocom 2002.
[5] H. Melvin and L. Murphy, “Time synchronization for VoIP Quality of Service,” IEEE Internet Computing, vol. 6,no. 3,May-June 2002, pp. 57-63.
[6] W. Jiang and H. Schulzrinne, “QoS measurement of Internet real-time multimedia services,” tech. report CUCS-05-99m, Columbia Univ., New York, Dec. 1999.
[7] R. Ramjee, J. Kurose, D. Towsley, and H. Schulzrinne, “Adaptive playout mechanisms for packetized audio applications in wide-area networks,” Proc. Conf. Comp. Comm. (IEEE Infocom), IEEE CS Press, Los Alamitos, Calif., June 1994, pp. 680-688.
[8] S. Moon, J. Kurose, and D. Towsley, “Packet audio playout delay adjust-ment:performance bounds and algorithms,” ACM/Springer Multimedia
Systems, vol. 6, pp. 17-28, January 1998.
[9] Y. Liang, N. Farber, and B. Girod, “Adaptive playout scheduling using time-scale modification in packet voice communications,” Proc. of
ICASSP 2001.
[10] Recommendation P.800, “Methods for Subjective Determination of Transmission Quality,” Aug. 1996.
[11] J. Daigle and J. Langford, “Models for analysis of packet voice com-munications systems,” IEEE Journal on Selected Areas in Comm., vol. SAC-4, no. 6, pp. 847-55, Sept. 1986.
[12] A. Clarke, “Modeling the effects of burst packet loss and recency on subjective voice quality,” Proc. of IP Telephony Workshop, Mar. 2001. [13] J. Janssen, D. De Vleeschauwer, M. Buchli, and G. Petit, “Assessing
voice quality in packet-based telephony,” IEEE Internet Computing, vol. 6,no. 3,May-June 2002, pp. 48-56.
[14] Recommendation G.113, “Transmission impairments due to speech processing,” ITU, Feb. 2001.
[15] W. Jiang and H. Schulzrinne, “Modeling of packet loss and delay and their effect on real-time multimedia service quality,” NOSSDAV 2000, Chapel Hill,NC, Jun. 2000.
[16] S. Voran, “Speech quality of G.723.1 coding with added temporal discontinuity impairments,” Proc. of ICASSP, May 2001.
[17] D. Mills, “Network time protocol: specification, implementation, and analysis,” IETF RFC 1305, Mar. 1992.
[18] W. Montgomery, “Techniques for Packet Voice Synchronization,” IEEE
Journal on Selected Areas in Comm., vol. SAC-1, no. 6, Dec. 1983.
[19] G. Almes, S. Kalidindi, M. Zekauskas, “A One-way Delay Metric for IPPM,” IETF RFC 2679, Sept. 1999.
[20] H.Schulzrinne, S.Casner, R.Frederick, and V.Jacobson “RTP: A Trans-port Protocol for Realtime Applications,” IETF RFC 1889, Jan. 1996. [21] H. Melvin and L. Murphy, “Implementation of a Hybrid Playout
