Simulation of Combined Rate-adaptive and Priority Marking Method

The modules described above can be integrated to support the simulation of the new com- bined quality of service control method. Each user’s packets are traced and recorded for evalu- ation. The packet size and packet loss information is used to process a reference speech to get a degraded speech. The degraded speech is then compared with reference speech using PESQ to get the evaluation result. The results of the simulation are discussed in the next section.

9.5

Results and Analysis

In order to investigate how the QoS control schemes affect perceived speech quality under different network conditions, we simulated different network congestion scenarios using the

9.5. Results and Analysis

NS-2 network simulator. The bandwidth of the bottleneck link was set to a fixed value (2Mbit/s) with a delay of 1ms. The number of the streams sharing the link was increased from a small number to a large number to simulate different congestion scenarios. The starting point was 70 streams sharing the bottleneck when there is no congestion at all. The number of users was increased from 70 to 140 in steps of 5. By reaching 140 users, almost every stream suffered from a very high loss rate and all the control methods were unable to cope with the impairments well. (Packet loss rate for non-control scheme was measured more than 40%). The latest investigation about PESQ method’s performance in high packet loss situation [71] suggested that MOS score is much lower from PESQ result so we stopped increasing the user number after 140 as the result is meaningless.

In order to compare the performance between the different QoS control schemes and a ’no control scheme’, we also implemented the priority marking, the rate-adaptive and no control schemes. For the priority marking and no control schemes, the send bit rate of the AMR codec was set to a fixed mode (12.2 Kb/s). For rate-adaptive-only control method, the bottleneck link was set to a non-DiffServ link with the same delay parameters and the rest of the system remained the same. The simulation was carried out using the same scenarios as described pre- viously. The number of simultaneous users was increased from 70 to 140 as before. Figure 9.7 compares the results for all four schemes.

The results show that for 70 simultaneous users, i.e. when there is no congestion, all four methods have the same performance. The MOS scores represent the highest score obtainable from an AMR codec.

In general, as shown in Figure 9.7, the drop of the speech quality follows the similar pattern for all four schemes because they were all suffering from the packet loss occured in the bot- tleneck link (see Figure 9.5). For the adaptive rate control scheme, the drop of speech quality is less steep compared with the ”non-control” scheme. This is because the MOS driven rate adaptation can choose the best-optimised AMR rate to minimise the affect of codec rate de- crease and packet loss increase. For the priority-marking scheme, the improvement over the

9.6. Summary 1 1.5 2 2.5 3 3.5 4 4.5 70 75 80 85 90 95 100 105 110 115 120 125 130 135 140 Number of user N M O S No control marking rate adaptive Combine

Figure 9.7: MOS vs. Number of user N for different control and non-control schemes)

non-control scheme is stable although not very significant. This is because although the Diff- Serv method can be used to treat different packets with different priority (i.e. loss rate), higher priority packets still have chance to be dropped, especially when the congestion is higher than CIR. From the figure, the performance of the new combined scheme is always better than those two different control schemes and the non-control scheme.

9.6

Summary

In this chapter, a new QoS control scheme has been proposed which combines the strengths of rate-adaptive and priority marking QoS control schemes and uses a predicted objective mea- sure of speech quality as a control parameter. We investigated perceived speech quality for dif- ferent QoS control schemes by integrating NS-2 network simulator with a real adaptive speech codec (the AMR codec) and a perceived quality evaluation system based on the ITU PESQ algorithm. We used the predicted perceived speech quality metric (measured by PESQ), in- stead of individual network parameters, to control the AMR codec’s send bit rate. Preliminary

9.6. Summary

results show that the new control scheme achieved the best perceived speech quality compared with rate-adaptive, priority marking and no control schemes in different network congestion conditions.

In future, the investigation can be extended to include the application of the combined control scheme in a TCP/UDP mixed environment. The effects of delay in the DiffServ model and the use of conversational speech quality (instead of listening quality) as metric to control AMR rate can be studied.

This work has led to another PhD project because it is of a major research interest in its own right.

Chapter 10

Internet-based Subjective Speech Quality

Measurement

10.1

Introduction/Motivation

From Chapter 5 to Chapter 9, the work is around the development of novel objective non- intrusive speech quality prediction methods/models and their applications in voice quality mon- itoring/prediction, buffer optimization and QoS control. As mentioned before, speech quality can be measured using either subjective or objective methods. Subjective measurement (e.g. MOS) is the benchmark for objective methods, but it is time consuming, and expensive. In this chapter, the existing subjective test methods are investigated and an efficient Internet-based subjective test methodology is proposed.

The traditional MOS test methodology has been in existence for about 20 years [133] and today its uses range from the assessment of codec quality to the assessment of VoIP network quality. The stringent test requirements for traditional tests have not changed (e.g. the use of a sound-proof room) in that time and are essential for a proper assessment of voice quality in many cases, e.g. quality assessment of codecs, as the difference between codecs may be subtle and difficult to detect. However, for VoIP applications, new impairments, such as packet loss, are much more perceptible than impairments from codecs. This has led us to investigate the possibility of conducting MOS tests under normal working/studying environments, as this is

10.1. Introduction/Motivation

more realistic and subjects are more relaxed. In a sound-proof room, some subjects may find it uncomfortable, psychologically, to carry out tests in the confined environments. This has led to an Internet-based subjective test methodology, which has the following advantages:

- It is closer to reality than the traditional method. Subjects remain in familiar environ- ments, e.g. an office or a laboratory, to carry out the test. This is clearly less stressful and the test can be done at the subject’s own pace.

- It is possible to organise subjective tests at more locations around the world.

- It allows easier access to a larger number of subjects (e.g. 40 - 80 subjects can be tested at the same time in one or two large rooms, e.g. a laboratory).

Overall, it has the benefits of efficiency, realism, wide access and ease of organisation. It can save money and time compared to P.800 [3]. Of course, the main disadvantage of Internet- based MOS test is the lack of a controlled testing environment (e.g. very low background noise) compared to P.800.

Two series of Internet-based MOS tests are carried out. The first one is without control (subjects did their own tests on their own computer, in their own office and at their own pre- ferred time slot). This is extended by introducing a measure of control to reduce the impact of different working environments on the results. In the Internet-based MOS test method, all subjects sit in a large project room which they use regularly. It is not a sound-proof room, but it is quiet and has Internet access.

In this chapter, the work on these two series of Internet-based MOS tests is presented. The uncontrolled Internet-based MOS test is described in Section 10.2 and controlled Internet- based MOS test is presented in Section 10.3. The test set-up and the quality evaluation between subjective tests and objective test methods are also given. Section 10.4 concludes the chapter.