Quality of Experience (QoE) is a topical issue for Internet video, particularly at the client- side. It has been defined as“the degree of delight or annoyance of the user of an application or service.” [119]. This simple definition belies the complex, multifaceted nature of actually measuring human perceptions of quality, especially for an application area as variable and as diverse as Internet video. In recent years, QoE has emerged as a multidisciplinary field with facets in mathematics, statistics, psychology, and engineering – all focused on understanding overallhumanperceptions and requirements of quality. However, it is widely acknowledged that there is no single, all-encompassing QoE metric for Internet video today [120]. There are several different, well-known methodologies, metrics and approaches which are used in different scenarios, but have various shortcomings. Careful consideration is often required in order to suggest the most useful QoE metric and methodology for a given scenario or use case. This ambiguity makes QoE a very vibrant area of research, with several periodicals, and even entire conferences and journals dedicated to it.
3.6. QUALITY OF EXPERIENCE 35
To the best of my knowledge, energy usage or greenness have never been considered as a parameter for Quality of Experience, despite all the attention. Using the mobile phone scenario as an example, the energy usage of a video will have a direct impact on the overall quality of experience that will be achievable by the device user. If the battery drains quickly and the user can no longer use the phone, then the overall Quality of Experience attained by the user will be much degraded. In Chapter 4, I introduce a novel means of combining energy usage into existing QoE metrics to derive a new, energy-aware QoE metric for video.
3.6.1
Objective Quality Assessments
In the ideal case, measuring the QoE of an application should involve assessments of real human perceptions of that application. Considering the scale and diversity of Internet video, this is often impracticable. Objectivemeasures of video quality offer relatively quick and repeatable means of evaluating quality of video through statistical and mathematical analysis which aim toapproximatethe human perception of video quality. The most often-used objective metrics in the literature today, are the Peak Signal to Noise Ratio (PSNR) and the Structural Similarity Index Metric (SSIM).
Mean PSNR is a very mature and popular metric used for measuring objective quality of video. PSNR is the ratio between the maximum possible power of a signal and the power of corrupting noise that affects the fidelity of its representation. The signal in the case of video, is an original, uncompressed video format - such as a raw YUV file, and the noise is the error introduced by the encoding process. For a video sequence, the mean PSNR across all the individual video image frames in the sequence is computed. As an original source has to be used in the comparison, PSNR is an example of a
Full-Referenceobjective metric.
PSNR is derived from the Mean Square Error (MSE) as follows:
MSE = ∑ n j=1∑mi=1(xi,j−yi,j)2 mn (3.1) PSNR = 10 log MAX 2 I MSE (3.2) where:
xis an uncompressed video image of sizem×npixels
36 CHAPTER 3. STATE OF THE ART: ENERGY AWARENESS IN VIDEO
MAXI is the maximum possible pixel value of the image (e.g. 255 for 8-bit video)
A higher PSNR generally indicates that the encoded video is of higher quality. For 8-bit video, typical values of acceptable to good quality lie between 30dB and 48dB. However, although PSNR is often used due its simplicity and speed, it is quite a controversial metric. Many authors have argued for and against [121] the correlation of PSNR with real human perception of video quality. Other authors have claimed it is only valid when comparing qualitywithincodecs [121]. Regardless of this criticism, it has been very well used in comparing different codecs in numerous studies. In any case, this further highlights the fact that there is no single all-encompassing metric for assessing video quality.
The Structural Similarity (SSIM) Index was proposed in 2004, and was designed to have a better approximation to human perception [122]. SSIM is a based on a human perception model that considers image degradation as perceived change in structural information between images. The SSIM index is calculated on variouswindowsof an image. The measure between twowindows xandyof common sizeN×Nis calculated as:
SSI M(x,y) = (2µxµy+ci)(2σxy+c2) (µ2x+µ2y+c1)(σx2+σy2+c2)
(3.3)
where:
xandyare two non-negative, spatially aligned image signals for comparison
µx is the mean ofx;
µythe mean ofy;
σx2is the variance ofx;
σy2is the variance ofy;
σxyis the covariance ofxandy;
c1= (k1L)2,c2 = (k2L)2are variables ;
k1 = 0.01 and k2 = 0.03 by default L is the dynamic range of the pixel-values (i.e.
2#bits per pixel−1 ) e.g. 28−1=255 for 8-bit video image; [122].
As the calculated SSIM index value approaches 1, the greater the degree of fidelity or the structural similarity of the encoded copy is to the original. Typical SSIM values for
3.6. QUALITY OF EXPERIENCE 37
acceptable to excellent quality will fall between 0.90 to 0.99.
In recent literature that presents objective measurements, both the PSNR and SSIM are calculated and presented [123, 124]. Both metrics are also widely used in commercial applications, such as by VoD providers, ISPs, telecommunications companies and cable/satellite TV, as these providers will want to continuously ensure that their video offerings are of acceptable quality. Additionally, there are several other objective quality metrics in existence - Multiscale-SSIM [125], Temporal Variation Metric (TVM) [126], Video Quality Metric (VQM), Video Multimethod Assessment Fusion (VMAF) [127] and others too numerous to cover in detail [128]. Recently, Dobrian et al. [129] presented metrics that quantified user engagement for Internet VoD using metrics such as QoS- related metrics such as join time, buffering ratio and network bitrate. Ammar et al. also suggest the use of network-level fairness as a QoE metric[130, 131]. In Chapters 4 and 5, I present a novel but straightforward means of combining any of these objective metrics such as SSIM and PSNR, with energy usage as a combined, energy-aware quality metric.
3.6.2
Subjective Quality Assessments
Subjective measurements of video quality offer a more detailed, human-centric approach to video quality assessment. Ultimately, videos will be viewed by any number of humans, ranging from the tens to the hundreds of millions. Therefore, it makes logical sense for these videos to be assessed, at least, by a small but representative subset of human assessors - rather than by synthetic, mathematical approximations of human perceptions of quality. This is why subjective measurements of video quality are often necessary.
Unfortunately, these subjective assessments with real human users are relatively expensive to perform in terms of time and money. A complete objective assessment on a single video can be completed in seconds using standard software and hardware. On the other hand, a comprehensive subjective assessment of that same video could take several man-hours to complete. For statistical significance, it is required that at least 4 human evaluators (must be involved in a subjective assessment of video [132]. However, in practice, most studies will involve 15 - 40 human evaluators [132]. Nevertheless, several subjective assessments of video have been, and will continue to be performed for research and commercial purposes, as they deliver a higher level of confidence and accuracy in video qualit assessments, when compared to objective measurements. As such, there are a plethora of subjective studies on video in the literature for a range
38 CHAPTER 3. STATE OF THE ART: ENERGY AWARENESS IN VIDEO
of purposes. The International Telecommunication Union (ITU) have published a number of recommendations on how such studies can be performed with scientific rigour [132, 133, 134]. These recommendations were authored by experts in the field and describe several different methods by which subjective studies of video quality can be performed. Some of these methods are: Absolute Category Rating (ACR), Degradation Category Rating (DCR), Pair Comparison (PC), Double Stimulus Continuous Quality Scale (DSCQS) Single Stimulus Continuous Quality Evaluation (SSCQE) and Subjective Assessment Methodology for Video Quality (SAMVIQ). The choice of which subjective method to use depends on the particular video application being evaluated e.g. interactive or non- interactive.
Many scientific studies have included the use of subjective assessments of video quality. Some have been performed to compare the performance of various codecs to one another [135, 136], while others have been done to investigate the impact of different network conditions or other impairments on the video streams [137, 138, 139]. With the recent emergence of adaptive video streaming systems like DASH over the last few years, a small number of researchers have performed subjective assessments specifically on DASH video. For example, Robinson et al. [140] performed one of the earliest published subjective assessment of HTTP adaptive streaming using the proprietary solutions that were available at the time (the experiment pre-dates the DASH standard. They evaluate these solutions on a range of network conditions including varied bandwidth, packet loss and latency profiles. Yitong et al. [141] also evaluate subjective user QoE for MPEG-DASH. They emulate network conditions according to collected traces to evaluate the human perceptions of the bandwidth adaptation behaviour of DASH, under pre-determined network conditions. Additionally, many VoD providers make use of simple rating schemes and feedback forms (e.g. through ’stars’ and ’likes’) to get information from their users on the quality of thier videos. While these methods may lack scientific rigour, the large sample sizes enable the collection of valid and useful insights.
In Chapter 6, I present the first user study that investigates consider human perceptions of energy-aware DASH adaptation, as well as user preferences and inclinations for energy savings while using Internet video.