QoS Measurements Methods and Tools

Contact: Jarmo prokkola [email protected] Tel: +358 20 722 2346 VTT Technical Reseach Centre of Finland Easy Wireless Workshop, IST Summit, Budapest, 05.07.2007

Network performance vs. QoS

•Network traffic measurements are employed to study the pure network performance

•Active measurements are performed by injecting traffic with known properties into the network

•Passive measurements consists of monitoring the existing traffic flow(s) at one or more points

•QoS measurements lie logically above network traffic measurements, and relate to the performance of networking applications.

•Objective QoS relates to something concrete and quantitative •Packet loss, Delay, Jitter, Connection break length…

•Subjective QoS corresponds to the service quality from the user perspective.

•Mean Opinion Score (MOS) tests are often used

•Subjective QoS can be estimated within certain limits from the basis of objective QoS (e.g., PESQ algorithm)

The purpose of QoS measurements & monitoring

•Answers to the question: How do the applications see the network? •Real-time QoS monitoring benefits

•Network administrators

•Network equipment manufacturers (testing) •Networking application developers

•Operators •End users

•Networking performance

•QoS aware applications (e.g., adapting video traffic flow) •Handover triggering

•Congestion control management

•Accurate QoS measurements can be used by •Network equipment manufacturers (testing) •Networking application developers

Common QoS Measurement Architecture

•QoS measurement architecture includes: •(a) Measurement points

•Located at network nodes (e.g., hosts, routers, firewalls) •(b) Traffic measurement tool

•Captures packets and collects

information of the desired traffic flow •(c) QoS analysis tool

•Analyzes the collected data and calculates the actual QoS statistics •(d) QoS monitor or database

•Analysis can be done in real-time,

and/or after the traffic traces have been collected.

•Analysis can be centralized to a specific server, or distributed to the network. Traffic measurement tool Measurement point (network node) Measurement point (network node) Traffic measurement tool QoS analysis tool

…

Figure 1. Functional decomposition of QoS measurement architecture.

Single-point measurements

•Measurement tool is operating in a single network node

•Round trip (RT) performance

•RT delay, RT jitter, RT packet loss •Service response time

•E.g., HTTP, or commonly TCP performance

•Single-point traffic statistics

•Offered traffic load, throughput, number of packets, packet sizes… •One-way performance can not be

measured

Round trip performance

Figure 2. A single-point measurement architecture.

Two-point measurements

•If the measurement tool is connected to both communicating nodes, this is called

end-to-end measurement.

•One-way performance can be measured separately for UL and DL

•Delay, jitter, packet loss, …

•Directional traffic flow performance, e.g., real-time traffic flows, UDP •Traffic statistics from both points

•Real-time monitoring requires control traffic

•Enables also two-way monitoring possibility in a single point

•Requires flow identification and timing

synchronization (for delay) between

measurement points

end-to-end performance Uplink (UL) performance Downlink (DL) performance

Figure 3. An end-to-end measurement architecture.

Multipoint Measurements

•End-to-end performance can be split to network segment performances.

•E.g., what is the access network performance vs. core network performance? •Enables bottleneck identification (e.g., how do individual network devices work?)

end-to-end performance

Network segment 1 performance Network segment 2 performance

•Measurement tool units could be passive

components, and e.g., network administrator could activate monitoring when needed

•Analysis and result

collection is challenging •Measurement units could

e.g., communicate with a centralized analysis tool

QoS measurement tools used in EW: MOSET

• MOSET (VTT’s freeware) is an active mobile service testing tool

• A testing application for mobile phone

• Execution of measurements / tests

• Instant results • A testing server

• Collect & manage results

• Distribute test applications and tests

• Service response time measurements • Connection establishment

• Retrieval of service content from network

• Measures HTTP/TCP performance from the user point or view

QoS measurement tools used in EW: QoSMeT

•QoSMeT (VTT) is a passive tool for measuring one-way end-to-end network QoS from the

application’s point of view

•Real-time measurements & accurate per packet statistics

•Is able to measure QoS of real-time

applications (e.g., VoIP, video conferencing), but other networking applications can be

measured as well.

•The tool works practically over any kind of network as long as IP is supported

•QoSMeT can be run in the same device with the measured application or within the network path at desired points

•Measures e.g.: Delay, Jitter, Packet loss, Connection break duration (e.g., during a

handover), Throughput and Offered load, and the volume of data sent/received

•GPS is used for clock synchronization

QoS measurement tools used in EW: M5 Multi-Analyzer

•M5 (NetHawk) enables protocol monitoring, call and session tracing, key performance indicator (KPI) analysis, QoS measurements, and radio

optimization measurements

•Monitors and analyzes UTRAN, GERAN, UMAN, IMS, WiMAX and core network interfaces at the same time.

•Supports all common transport techniques – STM-1/OC-3, STM-4/OC-12, E1/J1/T1 in different modes and Ethernet

•Product components: Standard PC, NetHawk Adapters & cables and NetHawk M5 analysis SW

•Enables multipoint flow measurements with QoSMeT

•QoSMeT handles the end-points, while M5 takes traces within the network path and decodes QoSMeT’s control packets

QoS measurement tools used in EW: Agent based solution

• The EW agent (Moviquity) is a software component attached to the terminal and server software:

• Measurement of main E2E QoS parameters (BW, delay, jitter, packet loss)

• Action control (switch to preferred network, codec change)

• Three kinds of agents are required:

• Server Agent (EWSA) including Central Agent • Network/Domain Agent (EWDA)

• User/Mobile Agent (EWMA)

• Via continuous feedback, it is possible to monitor actively the communications environment.

• Complete knowledge of the network and service status helps the server:

• To share and administrate resources with better efficiency

• Provide the maximum possible quality to the user taking into account the network the application service and the user profile

Figure 7. Agents taking delay measurements: It is seen that during video reception there is clearly more delay in the network.

EW Measurement Examples:

VHO Performance

•A simple handover scenario (VoIP application using Mobile IP):

LAN WLAN 3G WLAN LAN

•Great differences in delay and jitter between network technologies (e.g., 3G one-way-delay is only barely acceptable for VoIP)

•Connection break length is on the order of seconds during handovers

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.0 50.0 100.0 150.0 200.0 250.0 Time [s] De lay [ s] 0.01 0.1 1 10 0.0 50.0 100.0 150.0 200.0 250.0 Time [s] Con nect ion loss le ngt h [s ]

0.0 200.0 400.0 600.0 800.0 1000.0 1200.0 10 100 1000 10000 100000 1000000 10000000 UDP packet size [B], TCP downloaded file size [B]

G oodput [ kbit/s ] UDP HSDPA UDP 3G TCP HSDPA TCP 3G

UDP HSDPA Throughput

EW Measurement Examples:

HSDPA vs. WCDMA performance in a live network

•A great improvement with

HSDPA is got as compared to the basic 3G/WCDMA

•With UDP, goodput near to the operator’s limit (1 Mbit/s) is

achieved with reasonably small packet sizes.

•However, good TCP

performance still requires quite large packet size

• RT delay is high also in HSDPA, thus limiting the performance of TCP

=> A typical WWW user does not see the real potential of HSDPA!

Goodput

EW Measurement Examples:

Multipoint Measurements in VTT’

s CNL 3G Network

•Uplink delay behavior in different sections of the network

•Lightly loaded network, to minimize the effects of queuing delay

•Most of the total delay is caused by the radio

access part

•The effect of laboratory LAN is practically meaningless 0.000 0.010 0.020 0.030 0.040 0.050 0.060 0.070 0.080 0.090 0.100 0.0 5.0 10.0 15.0 20.0 Measurement time [s] De la y [s] End-to-end Radio access RNC simulator Ethernet (after RNC)

Conclusions

•The complexity of QoS measurements grow when increasing the number of measurement points

•While single point measurements are simple, the information provided is very limited

•Analysis of even a large monitoring network can be simple if only traffic statistics are needed. However, traffic flow behavior analysis is complicated and needs also clock synchronization if delay measurements are desired.

•QoS measurements play important role in EW, several tools are used, and new tools have been developed

•The tools are used in revealing the performance of the existing networks, giving feedback to applications, and in measuring the performance of the developed QoS methods.

•Interesting performance behaviors have been found in numerous measurements in EW, and several publications have been done.