Quality Of Experience

This paper presents a brief outline of Quality of Experience (QoE) in video traffic and describes how big data can provide a possible solution to the challenges in video QoE assessment. We have carried out an experiment that is used to gain an understanding of how we can apply the enormous amounts of data available to us when video is delivered through the Internet. Using this we place recommendations for the creation of future QoE models, particularly with big data in mind.

QoE, by those ITU-T definitions, involve degrees of delight or annoyance, can be understood as a final attitude of the user to the service influenced by expectations, beliefs, values and by social norms (influencing factors). This modeling is closely related to Fishbein and Ajzen's Theory of Rational Action (1975) which proposes that attitudes are permeated by beliefs and assessments and guide behavioral intent. This definition used by the ITU was inspired by the concept of QoE developed by the European Network on Quality of Experience in Multimedia Systems and Services, Qualinet (COST Action IC 10032). In 2011, The Group started to foster the scientific discussion about the definition of the term QoE and related concepts. As a result of this discussion, the “Qualinet White Paper on Definitions of Quality of Experience” (Le Callet, Möller, Perkis (eds), 2013) was compiled and published in its version 1.2, which presents, among other aspects, definitions and instrumental metrics on QoE. According to this conception, QoE is the: “(...) degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user‟s personality and current state. Degree of delight of the user of a service. In

The latest advancements in computer and communications-related technologies have brought to the wide consumer market many network solutions and a variety of net- work-enabled devices. Various types of networks enabled by different wired solutions such as DSL, ADSL, Ethernet or wireless such as WiFi (IEEE 802.11) and WiMax (IEEE 802.16) offer network connectivity with different characteristics. Available bandwidth, one of the most important characteristics, differs not only among different types of networks, but also for the same network type depending on users’ type and number, traffic type, pattern and size, environmental conditions (mainly for wireless), etc. This variability significantly affects transport capacity and quality of delivery regardless of content type. All these strongly influence users experience during their interaction with the systems, which is known as end-user Quality of Experience (QoE).

This research question is formulated by considering the performance of Quality of Experience to reduce interruption of video resolution issue during streaming as highlighted in RP2 in Table 1.1. This RQ2 is the primary to formulate the research objectives (RO2) of this project.

The UK government recently published ―Britain‘s Superfast Broadband Future‖ [Ref 1] that sets out its vision for the UK‘s future broadband infrastructure with virtually all homes having access to at least a 2Mbps broadband service by 2015. It envisions world class connectivity using various communications networks: fixed, fixed-wireless, mobile and satellite so that as many people as possible can be reached. The document recognises that satellite is a viable option for the most remote users and for those in some other ‗not-spots‘. However, there is an inevitable delay when communicating over geostationary satellites which can have a detrimental impact on the user experience, especially for bidirectional real-time or quasi real-time applications such as voice and gaming. Therefore alongside the policy document, Ofcom have commissioned this study which investigates the Quality of Experience (QoE) and user satisfaction when using broadband satellite communication services for applications such as online gaming, VPN, video/audio streaming and VoIP [Ref 2].

The literal Quality of Experience, i.e. Quality of Learn- ing Experience, has not directly been discussed in this paper; however, initial results of a related study show that the learning experience is largely determined by other factors such as the quality of design and delivery of the activity. As long as an experiment or service is usable, the impact of reduced technical performance on the quality of the learning experience is minimal. The study in this paper has demonstrated that usability thresholds could be identified by both, the user based as well as the automated tests. Intuitively, failure rate and MOS values are related as lost mouse clicks and keystrokes also introduce a level of annoyance for users. Future work will investigate and model this relationship in more detail.

Operators who optimize their video headends and TV portals on the device to improve the end user Quality of Experience see better user adoption, higher revenue potential and more positive brand awareness. While debates continue around the potential future for Mobile TV, it is clear that operators who have created attractive video services were able to generate substantial revenues and establish brand leadership.

ABSTRACT The quality of experience (QoE) perceived by users is a critical performance measure for Web browsing. ‘‘Above-The-Fold’’ (ATF) time has been recently recognized and widely used as a direct measure of user-end QoE by a number of studies. To reduce the ATF time, the existing works mainly focus on reducing the delay of networking. However, we observe that the webpage structures and content orders can also significantly affect the Web QoE. In this paper, we propose a novel optimization framework that reorders the webpage objects to minimize the user-end ATF time. Our core idea is to first identify the webpage objects that consume the ATF time but have no impact on the page experience and then change the positions of these objects to achieve the minimum ATF time. We implement this framework and evaluate its performance with popular websites. The results show that the ATF time is greatly reduced compared with the existing works, especially for complex webpages.

Abstract: Online learning tools have become important components of teaching and course delivery. This paper discusses the issues surrounding research into Quality of Experience (QoE) for online learning tools and how it relates to technical performance, Quality of Service (QoS). The relationship between QoE and QoS for online learning tools is often considered important for describing the optimal conditions for online learning environments. Such research largely ignores the vital issue of how learners differ from consumers in their use of information and communication technologies such as interactive multimedia environments. The implication of this difference for understanding technology use for learning is presented and the need for an empirical study to address this is argued for. A pilot was undertaken to further define the methodological requirements of conducting a study into the impact of system performance on QoE. The findings of the pilot study describe issues and implications for designing a research methodology which can begin the process of mapping the QoE to QoS relationship for online learning.

Abstract Online learning tools are widely used in engineering education. This includes traditional face-to-face, but also distance education. Since these tools rely on Internet connections, the performance of those connections (speed, latency) can impact on how learning tools are experienced by students. Quality of Service (QoS) describes technical performance parameters that reflect the quality of an Internet connection. Quality of Experience (QoE) on the other hand has been widely used to describe how users experience a particular service. In the context of this work, users are students undertaking learning tasks. While technical literature addresses QoE and educational literature discusses online learning, a gap exists describing the relationship of QoS and the quality of the learning experience. This work uses a mixed methods approach to address the research question: What dimensions of QoE of online learning can be affected by QoS? To answers this question, two groups of students were exposed to changing QoS conditions while they were undertaking an online learning activity using remote access technology. Both technical performance parameters, as well as, the impressions where recorded. Subsequently, a focus group was

The cheapest objective data available for an encoded video can be found directly in its bitstream. Even tough we do not know a-priori which of the bitstream’s properties are more, and which are less important, we can safely assume that they are related in some way to the perceived visual quality. How they are related, will be determined by data analysis. In this example, we use videos encoded with the popular H.264/AVC standard, currently used in many applications from high definition HDTV to internet based IPTV. For each frame, we extract 16 different features describing the partitioning into different block sizes and types, the properties of the motion vectors and lastly the quantization, similar to the metric proposed in [27]. Each frame is thus represented as 1 × 16 feature vector x. Note, that no further preprocessing of the bitstream features was done. Alternatively, one can also

In CF-Dash, we intentionally promote one specific profile within the cache. This is achieved by pushing clients’ play- ers to request one specific and commonly requested profile, we call it profile-limit. By doing so, the profile-limit will be populated within the cache and clients will be more likely to be served from the cache. From the network standpoint (i.e. Core network), our approach reduces the aggregated number of switching between qualities experienced by all clients and hence ensures stability of clients’ players. In CF-Dash, even though clients experience a high bandwidth, by default clients’ players never ask for profiles above the profile-limit. The profile-limit is chosen in such a way we guarantee a good user-engagement. Hence switching to the

The MLO QoE study is realized through the layered eval- uation methodology depicted in Figure 1. In general terms, the experience of a user with any application is conditioned by multiple influence parameters, including dimensions such as technical characteristics of the ap- plication, user personality and expectations, user demo- graphics, device usability, and usage context among oth- ers. Particularly when evaluating networking-based ap- plications, the influence of the network and its interplays with the particular application have to be linked to the user’s opinions, additionally identifying those perceiv- able performance parameters that are most relevant to the user experience. This mapping is realized by ana- lyzing and correlating the three layers depicted in Fig- ure 1: the network layer accounts for the influence of the network QoS parameters (e.g., network bandwidth, RTT, etc.); the application layer considers both the technical characteristics (e.g., codecs, video resolution, etc.) and the perceivable performance parameters of the applica- tion (e.g., response times, video and audio artifacts, etc.); finally, the user layer spans the user subjective opinions on the evaluated application (e.g., MOS values, accept- ability, etc.). The experimental evaluation conducted in this work was designed in such a way that all the three aforementioned layers could be properly measured.

Video has changed the main role of some Internet enabled devices to a simple TV screen. It has therefore, become crucial for video content providers to increase the user engagement and resource utilisation. The objective of initially developed models was to address compression artifacts. Frame freezing due to unreliable transmissions such as Real-time Transport Protocol (RTP) over User Datagram Protocol (UDP) has promoted more sophisticated models that can conceal some level of packet loss. More recently, progressive download over HTTP led to new models (Moller & Raake 2014). Reliable prediction models to assess video quality have become indispensable and have received a lot of attention by the research community during the last decade. The outcome of these efforts include a number of video quality assessment models with different levels of computational com- plexity and accuracy. In general, quality is assessed by the following principal methods (Moller & Raake 2014):

Improving the online video applications’ quality has a huge demand. Both the number of users and data demand per user are increasing. Therefore, the amount of traffic generated by on-line video platforms is huge and growing rapidly. For example, the global consumer video internet traffic is expected to be 80 percent of all consumer Internet traffic in 2019 up from 64 percent in 2014 [4]. The quality of a video streaming content is a function of both compression/streaming process and transmission conditions[5]. Therefore, increasing the video streaming quality depends on the improvements of both compression/streaming process and transmission conditions. Transmission conditions involve bandwidth, delay, jitter and loss[6]. In this thesis, we try to optimize the transmission conditions in order to obtain a better QoE for video streaming applications.

The substantive contribution lies in the identification and systematic evaluation of factors affecting the perceived acceptability of visual experience in mobile multimedia applications. These factors include size, resolution, content types, encoding bitrates, audio quality, text quality, shot types, and zooms. People expect a similar relative picture size (expressed by the viewing ratio - the viewing distance divided by the screen height) for mobile TV to what they are used to in typical living room setups. At typical viewing distances this required a minimum picture height of 4cm in an indoor lab setting and 4.5cm in a field setting on a train. The resolution of the content can be greatly reduced and content encoded from QCIF resolution (176x144 pixels) onwards can provide an acceptable experience at adequate sizes. Displays with high resolution affect the visual experience positively and the content can be up-scaled on them to the point at which the angular resolution of the content in the eye of the beholder is reduced to 12 pixels per degree (ppd). Along with previous research on viewing preference of TV content my findings suggest that an angular resolution between 14ppd and 11ppd represents a general threshold below which the acceptability of the visual experience of video rapidly drops off. However, for the best visual experience viewers of QCIF content prefer a higher angular resolution of around 19ppd. This angular resolution is also sufficient for the rendition of text (of 19 arc minutes height) included in TV content given sufficient encoding bitrates. When delivering TV content at QCIF resolution to mobile devices of at least 4.5cm screen height content adapatation in terms of the used shot types is not required apart from extreme long shots in football content. Players in this content should be at least 0.8 in height to be acceptable to football fans. The end user can achieve this by zooming (manually or software assisted) or by using devices with large enough screens; the content producer can help provide this acceptable height by creating zoomed-in content. The contribution of picture size to the visual experience is much larger than its contribution to video quality. Whereas an acceptable visual experience of mobile TV starts with 4cm picture height and 14ppd angular resolution, size only starts affecting the perception of picture quality once an angular resolution of 35ppd is reached. My findings refine the concept of QoE in the domain of mobile multimedia applications by promoting the concept of visual experience. My results point to a set of improvements that can be made to mobile multimedia services.

subjective scores is not performed. In other words, raw subjective scores are used in the subsequent analysis. After the subjective user study, two outliers are removed based on the outlier removal scheme suggested in [32]. After outlier removal, Z-scores are linearly rescaled to lie in the range of [0, 100]. The final quality score for each individual image is computed as the average of rescaled Z-scores, namely the mean opinion score (MOS), from all valid subjects. The final quality score for each individual image is computed as the average of subjective scores, namely the mean opinion score (MOS), from all valid subjects. Considering the MOS as the “ground truth”, the performance of individual subjects can be evaluated by calculating the correlation coefficient between individual subject ratings and MOS values for each image set, and then averaging the correlation coefficients of all image sets. The Pearson linear correlation coefficient (PLCC) and Spearman’s rand-order correlation coefficient (SRCC) are employed as comparison criteria, whose range is from 0 to 1 and higher values indicate better performance. They can be computed for each subject and their values for all subject are depicted in Fig. 3.2. It can be seen that each individual subject performs well in terms of predicting MOSs. The average performance across all individual subjects is also given in the rightmost column in Fig. 3.2. This provides a general idea about the performance of an average subject. Therefore, we conclude that considerable agreement is observed among different subjects on the perceived quality of the test video sequences.

Such baseline degradation depends on the specific task (interactivity degree and screen activity), and can be as high as 0.5 MOS for highly interactive graphical tasks (drag & drop[r]

Abstract: The evolution from traditional telecommunication networks towards NGN (Next Generation Networks) is enabling service providers to deploy a wide range of multimedia services such as Internet Protocol Television (IPTV), Video on Demand (VoD), and multiplayer games services, all on the same underlying IP network. However, managing the satisfaction level of customers has not been an easy task for network operators and service providers. In this paper, we analyze existing Quality of Experience (QoE) measurement approaches including Customer Experience Management (CEM) and Service Quality Management (SQM) schemes, which have been defined by Telecom Forums and standardizations bodies. A monitoring level inside Customer Experience Management System (CEMS) architecture is proposed in the context of IPNQSIS project to assess and quantify the user experience level, and accordingly adapt the network traffic.

From our theoretical analysis three testable implications follow. (i) As in the Klein–Leffler model, our study implies that price differences for alternative quality levels should significantly exceed the respective differences in production cost. (ii) If for all customers the willingness to pay for quality is convex as a function of quality (measured by produc- tion cost), quality polarization should be observed. That is, only extreme quality levels—“high” and “low”—should be available in the market. (iii) If an increase in quality raises all customers’ willingness to pay by more than it raises the respective production cost, prices of alternative quality levels below maximum quality should be a linear function of production cost. Moreover, these prices should be independent of demand shifts (which should affect only the quantities demanded and produced) and of changes in the intensity of competition due to alterations in the firms’ entry cost. The implication’s condition that relative to production cost customers have a sufficiently strong preference for quality, is frequently used in the literature on Bertrand equilibria in markets with vertical product differentiation (e.g., Shaked and Sutton, 1983, 1987; Sutton, 1986, 1991), and it is specified more precisely in Section 6 below. The condition is compatible with convex willingness to pay functions, but in that case intermediate quality levels should not be observed in the market because of implication (ii) above.