Dynamic Load Balancing and Node Migration in a Continuous Media Network

(1)

Dynamic Load Balancing and Node Migration in

a Continuous Media Network

Anthony J. Howe

Supervisor: Dr. Mantis Cheng

University of Victoria

Draft: April 9, 2001

Abstract

This report examines current technologies available in a continuous media network and presents two new areas. Current network structures include forward proxy caching, server farms, and content distribution net-works. Methods for broadcast distribution include controlled connection and one-way connection communication. Two important areas of research not addressed by current technologies are dynamic load balancing and node migration.

1 Introduction

Streaming media is found everywhere on the Internet! Radio stations use this technology to deliver their programs to a broader audience than can be reached by a local broadcast. Live broadcasts from public music concerts are streamed across the Internet. News web sites such as CNN.com deliver news through streaming video clips. Individuals are able to deliver their own mp3 collections to a world wide audience using Nullsoft’s SHOUTcast.

The increased application of streaming media brings increased demand. Technologies for continuous media on the Internet are rapidly evolving to meet this high demand. A streaming technology involves data compression algo-rithms, and the reliable delivery of data from the source to the end user. The Internet makes jitter free delivery of continuous data difficult due to unpre-dictable network congestion. Jitter is the variation of time between the delivery of a series of packets of continuous media [1]. For continuous media reliable delivery is not as critical as jitter free delivery. A continuous stream that expe-riences jitter will result in distorted audio or video for the end user.

This report examines current streaming delivery technologies and proposes and explains two important areas of research not addressed by current technolo-gies: dynamic load balancing and node migration.

(2)

2 Current Technologies

In 1995 Real Networks released a continuous media delivery application. This application delivered streaming media from a single server [2]. A single server limited the available audience size due to the performance of the server and the network bandwidth available. To reduce load on a central server three common network structures have evolved: forward proxy caching, server farms, and content distribution networks [3]. In addition the method of broadcast distribution has evolved. This section examines these network structures and streaming technologies and discusses current technologies that employ these ideas.

2.1 Forward Proxy Caching

Forward proxy caching attempts to bring continuous data closer to the end user. It has been in use for several years as front ends to web servers [3]. Figure 1 shows a forward proxy caching network. The nodes pass all their requests through the proxy before going to the origin. If the proxy is able to answer the request it returns the relevant information. The proxy provides faster response time to the node and reduces load on the origin. A disadvantage of proxy caches is that an uncommon request still has to go to the origin. Many uncommon requests from nodes around the Internet at one time could overload the origin.

origin

proxy cache

node node node

node's send requests to the proxy

Internet

first time requests are sent to origin

(3)

Real Networks has used this proxy caching technology in their RealSystem Proxy 8 [4]. The RealSystem Proxy 8 is able to handle both on-demand media and also deliver live broadcasts. For on-demand data, the first time media is requested it comes directly from the origin but is then stored in the cache of the RealSystem Proxy for any future requests. For live media a single stream from the origin is split and delivered to many requesting nodes. This saves bandwidth since many nodes access the stream from the local proxy as opposed to each accessing a separate live stream from the origin.

2.2 Server Farms

Server Farms rely on an intelligent switch to evenly distribute requests among a group of computers hosting the same information [3]. They appear to the user as a single origin. Figure 2 shows a server farm. Nodes are distributed evenly among the servers. A server farm provides redundancy. If any server fails requests are just routed to other servers. An advantage to this type of network is that all the servers are in one location and can be managed easily. A disadvantage to a server farm is in the delivery of continuous data. A server farm still has the problem of jitter free delivery to nodes many hops away.

source server

node node node

Internet source server source server Intelligent Switch Server Farm

Figure 2: An example of a server farm

Nullsoft’s SHOUTcast (http://www.SHOUTcast.com) technology is prone to this problem. The goal of SHOUTcast is to enable anyone to be able to deliver streaming data to the Internet [5]. Their product is not concerned with splitting streams and using caches at the edges of the network but only the

(4)

delivery from the source to mirror servers. Figure 3 shows a sample SHOUTcast network. A generator node sends streams to mirror servers. These mirror servers deliver the content to end users. There is no intelligent switch to these servers and it is up to the end user to choose a mirror. Since the end user may be many hops away from the mirrors, jitter free delivery of data from the mirrors may be impossible due to unpredictable network congestion.

SHOUTcast mirror

node node node

SHOUTcast mirror SHOUTcast mirror SHOUTcast generator

Figure 3: A Sample Shoutcast network

2.3 Content Distribution Networks

A content distribution network takes both the advantages of server farms and caching proxies. Figure 4 shows a content distribution network. Replicas of the origins data are transferred to servers named surrogates located geographically far apart. When a node requests data they first communicate to the request routing system that forwards the node to the “best” surrogate. The measure of “best” can be derived from geographic location and current network load and congestion. By locating the surrogates in various geographic locations nodes have a higher chance of experiencing quick and jitter free delivery of data from a close surrogate.

An advantage of the content distribution network is that the origin can be decoupled from the delivery network. The owner of the origin can out-source the management of the delivery of data to another organization. Or-ganizations such as Digital Island (http://www.digitalisland.com) and Akamai

(5)

Calgary surrogate

node node node

Toronto surrogate Vancouver surrogate Request Router Distribution System origin 1 1 1 2 2 2 3 3 3

Figure 4: A content distribution network

(http://www.akamai.com) currently provide distribution infrastructures spread throughout the world to deliver discrete and continuous data.

RealNetworks (http://www.realnetworks.com) provides a distribution tech-nology called iQ that can be used in the above global networks to distribute continuous data. RealNetworks iQ technology is concerned with the delivery of data streams from a source to servers that may be located throughout the world such as in a Content Distribution Network [6]. RealNetworks iQ allows for continuous media distribution or live broadcast distribution. It uses redun-dant methods to propagate streams to surrogate servers located throughout the world. Redundant methods include sending multiple streams on multiple net-works and then merging them back at the source, and forward error correction. The servers in the iQ network act as peers and are able to share capacity among each other and react in case of a network failure [6].

2.4 Two Methods for Broadcast Distribution

The white paper Live Broadcast Distribution with RealSystem Server 8 [6] by RealNetworks discusses two methods for broadcast distribution. The earlier method used two TCP connections and a UDP connection to deliver a data stream. The new method uses just one UDP connection for delivery of a data stream from a source to the destination.

(6)

method [6]. Figure 5a shows this method. The UDP connection is used for the main distribution of live data. The return channel is used for notifying the source of lost packets. The TCP control channel is used for the control of the data stream. Using the extra lines for lost packets and control introduces a latency in the data stream.

source receiver

persistant TCP control channel unidirectional data channel (UDP) return channel for resent requests

source unidirectional data channel (UDP) receiver a

b

Figure 5: Two methods for broadcast distribution

To reduce the latency in the data stream the newer broadcast distribution method uses just one UDP connection. To add redundancy to this UDP line, the stream is encoded using forward error correction (FEC) [6]. One example of forward error correction is using Solomon codes. Figure 5b shows this method. In addition to RealNetworks another company developing a form of FEC is Digital Fountain (http://www.digitalfountain.com/).

3 Dynamic Network Load Balancing and Node

Migration

The above technologies do not address two important areas for the distribution of continuous data: dynamic network load balancing and node migration. The first area of distribution not addressed is the dynamic addition more surrogate servers to the network when demand for continuous data is high and dynamic removal of surrogates when demand for continuous data is low. Reduction of surrogates means that nodes will have to be consolidated to the remaining surrogates; this is the second important area of distribution of continuous data. Both of these areas will add more scalability and jitter free performance to the above two technologies.

(7)

3.1 Dynamic Network Load Balancing

Figure 6 shows a Content Distribution Network with surrogate servers. Two continuous data sources are distributed to master surrogates on the network by the distribution hubs. The distribution hubs may broadcast the continuous data to the master surrogates using multicast communication. The master surrogates may further distribute the data to slave surrogates. A grouping of a master surrogate and slave surrogates is known as a surrogate group. Network nodes initially connect to one of the surrogates of the ”best” surrogate group. A definition of best may be not only be geographically close but also refer to the status of the network. A surrogate group that is further away geographically may be chosen for a node if there is network congestion between the node and its closest surrogate group.

dh dh S_m Ss Ss S_m Ss Ss src Source Network src Content Distribution Network Internet dh - distribution hub Sm - master surrogate server

src - source server

S_s - slave surrogate server

Figure 6: Distributing Streaming Content from source to surrogate to end user

A surrogate group must promote or bring online another slave surrogate when the group reaches capacity. All additional requests for continuous data will be forwarded to the newly promoted slave surrogate. The newly promoted slave surrogate will receive a data stream from a surrogate master. Figure 7 shows the addition of a surrogate as the network capacity increases for a surrogate group. Each surrogate can host a maximum of two nodes. In Figure 7a the network is at capacity and must promote the available offline surrogate. In Figure 7b the extra surrogate has been promoted and waits for a node connection. The next requesting node connects to the new surrogate in Figure 7c.

(8)

S_m - master surrogate server S_O - offline surrogate server

a b c S_m S_O _S Sm s S_m S_s

S_s - slave surrogate server

Figure 7: The promotion of an offline surrogate

will be demoted or taken offline. When this happens all the nodes on the demoted slave surrogates must be consolidated onto the remaining surrogates. Node migration is discussed in section 3.2.

3.2 Node Migration

Node migration is essential in consolidation of surrogate servers or for dynamic load balancing. A requirement of node migration is to ensure the continuous data is delivered such that the end user does not notice an interruption in the media. Node migration requires changes of buffer sizes on the node, and a swapping protocol. Each node requires a buffer to smooth out jitter caused by unpredictable inter-packet delay.

3.2.1 Changing Buffer Sizes

Since all surrogates receive the original streaming source at different latencies hosts must adapt their buffer sizes accordingly to continue to deliver media to the end user without interruption. Latency is not observable from a user point of view except at connection time. At connection time the latency is caused by end-to-end transmission delay and setup time.

When a node moves to a surrogate of greater latency from the original source than its current surrogate then its buffer will shrink. When a node moves to a surrogate of less latency from the original source than its current surrogate then its buffer will grow. The details of these findings can be found in the report

Buffer Management in a Continous Content Distribution Network by Howe [7].

3.2.2 The Swapping Protocol

When a node swaps from a surrogate of greater latency than the new surrogate, the node must catch up its delivery to equal that of other nodes connected to the same surrogate. If all the nodes connected to the surrogate are receiving the broadcast from a multicast then the recently swapped node must use a merge

(9)

method similar to the one described in the report The Split and Merge Protocol

for Interactive Video-on-Demand by Liao et al [8].

4 Conclusion

Two important areas of continuous media distribution are not addressed by cur-rent technologies. These two areas include dynamic network load balancing and node migration. Dynamic network load balancing and node migration will aid in scalability and jitter free delivery of continuous content distribution networks. Both of these areas require further investigation.

References

[1] G. Coulouris, J. Dollimore, and T. Kindberg. Distributed Systems: Concepts

and Design. Addison-Wesley, third edition, 2000.

[2] Rob Glaser. Realsystem iQ transforming digital media delivery. Public Announcement Video Stream, http://www.realnetworks.com/realsystem/, December 2000. Date Viewed: April 5, 2001.

[3] G. Tomlinson M. Day, B. Cain and P. Rzewski. A model for content in-ternetworking. Technical report, Network Working Group Internet-Draft, February 2001.

[4] Realsystem Proxy 8 overview. RealSystem iQ Whitepaper, RealNetworks, http://www.realnetworks.com/realsystem/, December 2000.

[5] SHOUTcast online documentation. Nullsoft,

http://www.SHOUTcast.com/support/docs/. Date Viewed: April 5, 2001.

[6] Live broadcast distribution with Realsystem Server 8. RealSystem iQ Whitepaper, RealNetworks, http://www.realnetworks.com/realsystem/, December 2000.

[7] Anthony J. Howe. Buffer management in a continuous content distribution network. Technical report, University of Victoria, March 2001. Draft. [8] Wanjiun Liao and Victor O. K. Li. The split and merge protocol for

inter-active video-on-demand. IEEE MultiMedia, 4(4):51–62, October–December 1997.