1
Internet Video Streaming and Cloud-based Multimedia
Applications
Yifeng He, yhe@ee.ryerson.ca Ling Guan, lguan@ee.ryerson.ca
Outline
• Internet video streaming
– Overview – Video coding
– Approaches for video streaming over Internet
• Cloud-based multimedia applications
– Overview
– Virtual Machine (VM) Allocation
– Workload scheduling
Overview of video streaming
• Video streaming is a real‐time application
– Live video streaming
• E.g., live broadcast of Olympic games – Pre‐encoded video streaming
• E.g., YouTube
3
Overview of video streaming
• Objective quality evaluation – Peak Signal‐to‐Noise Ratio (PSNR)
– Mean Squared Error (MSE) is distortion
• Encoding distortion
– Caused by quantization in video encoding
• Transmission distortion
– Caused by packet loss rate (PLR)
» Congestion loss
2552
Overview of video streaming
• Video streaming system
– Sender
• Capture, and encode the video
– Channel
• Wired or wireless channel
– Receiver
• Decode, and playback the video
Camcorder
Sender
Receiver Channel
Encoding
distortion Transmission distortion
5
Overview of video streaming
Original YUV before encoding Reconstructed YUV after
Overview of video streaming
• Protocol stack
Channel
Encoder Video Decoder
Video
Video
Video
Sender Compression domainReceiver
Transport domain
7
Overview of video streaming
• RTP (Real‐Time Transport Protocol): carries the video payload
– Payload type identification (e.g., video or audio) – Sequence numbering
– Time stamping
• RTCP (Real‐Time Control Protocol): provides feedback on the quality of the packet delivery
– RTCP monitors the quality of delivery, such as packet count, packet loss, delay.
Overview of video streaming
• Objective:
– Improve the received video quality
• Reduce the encoding distortion
– Rate‐distortion tradeoff – Error resilience
• Reduce the transmission distortion
– Quality of Service (QoS) mechanism for video streaming, to reduce the delay, packet loss.
9
Video coding
• Input raw video
– Resolution – Color space
• YCbCr, (also called YUV) – Luma: Y (brightness) – Chroma: Cb, Cr (colors)
– Sampling format
• 4:2:0
Video coder
Output compressed bit stream, bit rate = 1,000,000 bps Input raw video
(Foreman CIF 420:
bit rate=36,495,360 bps)
Video Frame Coding Techniques
I P P P P P P
I B B P B B P
11
• Approaches for video streaming over Internet
• Unicast-based techniques
– Centralized approach – Proxy approach
– Content delivery network (CDN) approach
• Multicast-based techniques
– Network-layer multicast
– Application-layer multicast (tree-based peer-to-peer live streaming systems)
• Approaches for video transmission over Internet
• Unicast-based techniques – Centralized
approach
13
Centralized approach
Proxy approach
• Approaches for video transmission over Internet
• Unicast-based techniques
– Proxy approach
15
Content Delivery Network (CDN) approach
• Approaches for video transmission over Internet
• Unicast-based techniques – Content delivery
network (CDN) approach
• Approaches for video transmission over Internet – Multicast-based techniques
– Network-layer multicast
Network-layer multicast
• Approaches for video transmission over Internet
– Apply P2P technology into video streaming applications
• P2P technology is a promising approach to build distributed and scalable network applications
• P2P network systems
– P2P file-sharing systems» For example, BitTorrent – P2P streaming systems
» P2P live streaming systems
• For example, PPLive
» P2P Video-on-Demand (VoD) systems
– Different between P2P file-sharing and P2P streaming
» P2P file sharing: play after downloading the whole file
» P2P streaming: play while the video file is being downloaded
– Different between P2P live streaming and P2P VoD
» P2P live streaming: The video playbacks on all users are synchronized
» P2P VoD: The playbacks of the same video on different users are not synchronized
17
• Approaches for video transmission over Internet
The exchanges of the blocks between two peers in different P2P systems: (a) P2P file- sharing systems, (b) P2P live
• Approaches for video transmission over Internet
– Multicast-based techniques
– Application-layer multicast (tree-based peer-to-peer streaming systems)
– Tree construction
19
Single-tree based P2P live streaming system Leaf node
Internal node
oDrawbacks
1. Departure of a peer causes the isolation of all of its descendants from the video source
2. All the leaf nodes do not contribute their uploading bandwidths
• Approaches for video transmission over Internet – Mesh-Based P2P Live Streaming Systems
4
4
4
Outline
• Cloud-based multimedia applications
– Overview
– Virtual Machine (VM) Allocation – Workload scheduling
21
Overview
• Cloud computing
– Enable convenient, on‐demand network access to a shared pool of configurable computing resources (e.g., virtual machine (VM), storage)
– Cloud computing models
• Infrastructure as a Service (IaaS)
– Examples: Amazon EC2• Platform as a Service (PaaS)
– Example: Google App Engine• Software as a Service (SaaS)
Overview
• Cloud‐based multimedia application
– Users:
• Send request to MSP and get result
– Multimedia Service Provider (MSP)
• Deploy multimedia services on VMs leased from cloud provider
• Offer multimedia services to users
– Cloud computing provider
• Provide VMs
23 Illustration of cloud-based multimedia service
VM allocation
• Virtual Machines (VMs)
– Resources in cloud are managed as VMs
– Different configurations have different service rates and different price rates
• Pricing Strategies
– On‐Demand Pricing Strategy
• No long‐term commitments
• Add more VMs on demand
• High hourly rate
– Reservation Pricing Strategy
• A low, one‐time, upfront payment for a VM instance
• Reserve it for a one or three year term
• Significantly lower hourly rate
VM allocation
25 The SaaS architecture for multimedia application providers
VM allocation
• VM allocation problem
– To optimize virtual machines allocation for each application to achieve the minimal resource cost for multimedia application providers, subject to the round‐trip‐time (RTT) requirements for all users.
• Proposed scheme:
– Optimize required VMs for each kind of application
– Use Round‐Trip Time (RTT) as QoS measurement
VM allocation
• Problem formulation
27
Reserved VMs cost
On‐demand VMs cost
Constraint on round trip time
Constraint on stable queuing
Constraint on reserved VMs amount
Workload scheduling
• Workload scheduling
– User‐level workload scheduling
• The requests for one application are distributed to different VMs for workload balancing
– Task‐level workload scheduling
• Each application is decomposed into multiple tasks
• The goal of task‐level scheduling is to assign tasks to
VMs so that the total execution time can be minimized
Workload scheduling
• User‐level workload scheduling
– Problem statement
• To optimally schedule users’ requests to different classes of VMs such that the response time can be minimized.
29
Response Time Minimization Problem
Mean response time Subject to
Resource constraints;
All requests have to be served
Workload scheduling
• User‐level workload scheduling
The workload scheduling model for multimedia cloud.
scheduler
Workload scheduling
• User‐level workload scheduling
– Problem formulation
– Analytical solution
31
Workload scheduling
• Task‐level workload scheduling
– A directed acyclic graph (DAG) is introduced to represent the precedence constraints among tasks
An illustration of video retrieval framework
Workload scheduling
• Task‐level workload scheduling
– Problem statement
• To optimally schedule tasks to different VMs such that the total execution time of the application is minimized, subject to the precedence constraints and the cost constraint.
33
Workload scheduling
• Task‐level workload scheduling – Problem formulation
• In a mixed structure, some tasks can be processed in parallel, while some tasks have to be processed serially.
• The DAG of mixed structure has multiple paths from the source to the sink. The total execution time is determined by the time taken by the longest path.
Every task should be assigned to one Objective: total execution time.
References
Internet video streaming
• D. Wu, Y. T. Hou, W. Zhu, Y. -Q. Zhang, and J. M. Peha, “Stream video over the Internet: approach and directions”, IEEE Transactions on Circuits and Systems for Video Technology, Vol. 11, No. 3, pp.282 – 300, Mar. 2001.
• T. Wiegand, G. Sullivan, G. Bjøntegaard, and A. Luthra, “Overview of the H.264/AVC Video Coding Standard”, IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, n0.7, 2003.
• Y. He and L. Guan, “Peer-to-peer streaming systems”, in Intelligent Multimedia Communication:
Techniques and Applications. Changwen Chen, Zhu Li, and Shiguo Lian (editors), Springer-Verlag Berlin Heidelberg, SCI 280, pp. 195–215, 2011.
http://www.ee.ryerson.ca/~yhe/my%20papers/BookChapter/Chapter_Springer_P2P_2009.pdf
Cloud-based multimedia applications
• X. Nan, Y. He, and L. Guan, “Optimal allocation of virtual machines for cloud-based multimedia applications”, in Proc. of IEEE International Workshop on Multimedia Signal Processing (MMSP), pp. 175- 180, Banff, Canada, Sep. 2012. (VM allocation)
• X. Nan, Y. He, and L. Guan, “Optimization of workload scheduling for multimedia cloud computing”, in Proc. of IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2872-2875, Beijing, China, May 2013. (User-level workload scheduling)
• X. Nan, Y. He, and L. Guan, “Optimal task-level scheduling for cloud based multimedia applications”, in Proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 3771- 3775, Vancouver, Canada, May 2013. (Task-level workload scheduling)
35