2 METHODOLOGY
The methodology is based on the performance of end-to-end automatic testes, thus making it possible to identity the quality of service on the field, giving the most realistic perspective of the networks’ performance as possible, from the user’s standpoint.
Measurement collection is made using drive-testes. Besides providing an assessment from the user’s standpoint, this approach makes it possible to have the test made independently from the correct functioning of the networks themselves.
On the other hand, the use of a sole testing system to asses the services provided by the three mobile networks, makes the results highly comparable, regarding time and space.
This approach also considers zones with bad or non-existent coverage in the assessment of mobile networks’ quality of service.
2.1 FUNDAMENTALS
This study’s methodology is based on three basic aspects:
a) End-to-end measurements: measurements reflect all aspects that impact a quality of a service;
b) Impartiality: Measurements are carried out under equal terms for the three operators (OPTIMUS, VODAFONE and TMN);
c) Objectivity: Tests are carried out in a totally automatic way, eliminating the subjectivity inherent to human intervention or decision.
2.2 QOSINDICATORS
From the user’s standpoint, the use of mobile services represents the following stages (different features of the Quality of Service):
a. Network Access – Shows that the mobile network is present and that it is possible to use the services (usually it corresponds to the indication of the network’s name on the screen of the
This information is also available in Anacom's Internet website in:
terminal equipment);
b. Service Access – When the user intends to use a service, the mobile operator should provide the access to that services the fastest way possible (e.g. to establish a voice call);
c. Service Integrity – Corresponds to the Quality of Service (QoS) during its use (e.g. Audio Quality during a voice call);
d. Service Consistency – Corresponds to the way how the use of the service is ended (according or not to the user’s will).
The mains Quality of Service Indicators were analyzed for each of the QoS features.
2.2.1 SERVICE INDEPENDENT
2.2.1.1 AVAILABILITY OF THE RADIO-ELECTRIC NETWORK (COVERAGE)
Probability of the mobile services being available to a user (network coverage).
[ ]
100% ts Measuremen of No. Total Services Mobile Available with ts Measuremen of No. % Network electric -Radio the of ty Availabili = ×Mobile services are considered to be available when the radio-electric signal shows values above the minimum levels that make its use possible.
The testing and measurement system used, through a RF Scanner, continuously measures each network’s signal levels. These measurements are geographically referenced, thus allowing for their representation on maps, making it easy to visualize the coverage levels of mobile networks on the routes under study.
Table 1 – Coverage Levels
Coverage GSM WCDMA
Good RxLev ≥ −85 dBm CPICH RSCP ≥ −95 dBm
Acceptable −95 dBm ≤ RxLev < −85 dBm −105 dBm ≤ CPICH RSCP < −95 dBm Bad −110 dBm ≤ RxLev < −95 dBm −115 dBm ≤ CPICH RSCP < −105 dBm Non-existent RxLev < −110 dBm CPICH RSCP < −115 dBm
2.2.2 TELEPHONY SERVICES
2.2.2.1 SERVICE ACCESSIBILITY (VOICE OR VIDEO-TELEPHONY)
Probability that the user has of having access to the service (voice or video-telephony), i.e., success probability when establishing a (voice or video-telephony) call.
It is considered that a call was “Established with Success” if it reaches the called terminal (on hears the “calling signal” on the calling terminal).
[ ]
100% Calls Establish to Attempts of No. Total Success with d Establishe Calls of No. % ity Accessibil Service = ×2.2.2.2 TIME TO ESTABLISH A CALL (VOICE OR VIDEO-TELEPHONY)
Period of time elapsing from the sending of a complete destination (destination telephone number) and the establishment of the call.
[ ]
1 2- t t s Call Establish to Time =t1 – moment when the user presses the send button.
t2 – moment when the call is successfully established
(one hears the “calling signal” on the caller terminal).
2.2.2.3 CALL TERMINATION RATE (VOICE OR VIDEO-TELEPHONY)
Probably of a call being maintained, after being established, during a given period of time, ending normally, i.e., according to the user’s will.
[ ]
100% Success with d Establishe Calls of No. n Terminatio Normal wirh Calls of No. % minationRate = × CallTer2.2.2.4 CALL AUDIO QUALITY (VOICE OR VIDEO-TELEPHONY)
This indicator quantifies how well the conversation is perceived during a (voice or video-telephony) call. Only calls with normal termination are considered for the reckoning of this indicator.
) f(MOS_ B side C ) f(MOS_ A side C LQO LQO = = ality allAudioQU ality allAudioQu
side A – refers to the quality perceived on the side of the mobile terminal (which is moving on the locations under analysis).
side B – refers to the quality perceived from the side of the ISDN fixed terminal, in the case of voice calls, or from the side of the mobile terminal,
motionless, in the case of video-telephony calls.
MOS_LQO – Perceived Audio Quality quantification scale (Mean Opinion Score_Listening Quality Objective).
PESQ1 and SQuad-LQ2 where the reckoning algorithms used for voice and video-telephony calls, respectively.
The MOS scale quantifies the effort that it takes to understand a conversation. Its value is 0 (zero) when there is no communication and 5 (five) when the communication is perfect. Value “zero” never shows on the results since they only consider situations where the connection was established and maintained during a given period. “Five” also never shows on the results because the CoDec3, used by mobile networks, do not make possible the existence of such high voice or video quality values (the voice or video quality reached with the usually used CoDec presents MOS values lower than 4.5).
Table 2 – MOS Scale
MOS Quality 5 Excellent 4 Good 3 Acceptable 2 Poor 1 Bad
2.2.2.5 VIDEO-TELEPHONY CALL VIDEO QUALITY
Quantifies the communication’s video quality using the VQuad4 algorithm developed by SwissQual. Only calla that ender normally are considered in the reckoning of this indicator.
) f(MOS Chamada de Vídeo Qualidade =
MOS – Perceived video quality quantification scale (Mean Opinion Score_Video Quality Objective).
The MOS scale present the levels already indicated on Table 2.
1 PESQ – Perceptual Evaluation of Speech Quality. Recommended by ITU-International Telecommunications Union (ITU-T Recommendation P.862
(02/2001); ITU-T Recommendation P.862.1 (11/2003)).
2 SQuad-LQ – SwissQual’s speech quality algorithm for Listening Quality. 3 CoDec – Codifier/De-codifier.
2.3 MEASUREMENT PROFILES5
Tests are conducted in a fully automatic way and using the Seven.Five system (there is no human intervention or decision during the conduction of a test).
Measurements are carried out in moving vehicles and with outdoor antennas (without gain);
Voice tests are made with manual selection of the 2G (GSM) infrastructure, while video-telephony tests are made with automatic selections of the 2G or 3G (GSM/UMTS) infrastructure;
All collected parameters are geographically referenced and can be later represented on digital cartography.
2.3.1 COVERAGE
Network coverage assessment is made by the continuous measurement of the signal levels of each mobile network, along each analyzed route.
Measurements are made through a RF Scanner, so the signal levels measures correspond to the effective levels on each point.
Signal samples are collected from all the radio-electric channels used by the three mobile operators, approximately every second for the GSM technology, and every two and a half seconds of the WCDMA6 technology.
These signal samples sample are later analyzed and only the best results obtained on each measurement point are considered in the GSM coverage and WCDMA coverage indicators.
2.3.2 TELEPHONY SERVICES 2.3.2.1 VOICE SERVICE
To analyze the voice service, test calls are made between mobile terminal equipment and terminal
5 The measurements profile presented here are supported on the technical specifications ETSI TS 102 250, namely on part 5 (ETSI TS 102 250-5 V1.3.1
(2005-11)), and ETSI EG 202 057, namely parts 3 and 4 (ETSI EG 202 057-3 V1.1.1 (2005-04) and ETSI EG 202 057-4 V1.1.1 (2005-10)).
equipment of a fixed network (ISDN). These calls are named MOC when they are originated on the mobile network and terminated on the fixed network, and MTC, when they are originated on the fixed terminal and ended on the mobile terminal.
The mobile terminal equipment moves along the studied route.
The time frames for making the calls are started simultaneously for the three operators.
After the call is established, the communication’s audio quality is analyzed, on both ends, according the
PESQ algorithm. Test Parameters:
► Relationship between MOC/MTC: 1/1; ► Duration of the test calls: 120 seconds; ► Time frame for making a test call: 180 seconds; ► Maximum time to establish a call: 20 seconds.
2.3.2.2 VIDEO-TELEPHONY SERVICE
To analyse the video-telephony service, test calls are made between mobile terminal equipment. One of the terminals is kept motionless, on a location with proper coverage, while the other terminal moves along the location under analysis. These calls are named MOC when they are originated on the mobile network that is moving and terminated on motionless terminal. On the opposite situation, the calls are named MTC.
The time frames for making the calls are started simultaneously for the three operators.
After the call is established, the communication’s audio quality is analyzed, on both ends, according the
SQuad-LQ and VQuad algorithms (algorithms owned by SwissQual, the supplier of the used testing a
measurement equipment).
Test Parameters:
► Relationship between MOC/MTC: 1/1; ► Duration of the test calls: 120 seconds; ► Time frame for making a test call: 210 seconds;
► Maximum time to establish a call: 20 seconds.
► Maximum time to establish the audio/video call: 20 seconds.
2.4 TEST/MEASUREMENT AND POST-PROCESSING SYSTEM
The Seven.Five/NetQual system, recently purchase by ANACOM, was used for measurements on the field and for their post-processing.
