Scalable Service Oriented Architecture for Audio/Video Conferencing

(1)

Scalable Service Oriented Architecture

for Audio/Video Conferencing

By

Ahmet Uyar

(2)

Outline

Research Issues

Criteria for videoconferencing systems

Overview of current videoconferencing systems

Overview of GlobalMMCS architecture

NaradaBrokering overview and additions

Performance tests for audio/video delivery

(3)

Research Issues

We investigate the question of how to develop scalable and

universally accessible videoconferencing systems over Internet.

We propose using publish/subscribe event broker systems for the

distribution of real-time audio and video streams in videoconferencing

sessions and investigate the issues pertaining to scalability,

performance, data representation, meeting management and media

processing services.

Since real-time audio/video delivery requires low latency and high

bandwidth, we investigate the performance and the scalability of this

software based messaging middleware extensively.

We propose service oriented architecture for videoconferencing. We

identify the tasks performed in videoconferencing sessions and

provide independently scalable components for each task. We

identified three main tasks in videoconferencing sessions:

audio/video distribution

media processing

(4)

Criteria for Videoconferencing Systems

We identified the following criteria for

videoconferencing systems:

Scalability

Security

Traversing through firewalls, proxies and NAT

Supporting heterogeneous clients

(5)

Videoconferencing Standards and Systems

Multicast based systems

AccessGrid

H.323 based systems

Polycom

CUseeMe

(6)

Multicast Based Systems

AccessGrid is the most commonly used room based

videoconferencing system for group communications.

Scales well.

Difficult to provide security services. No authority to manage

multicast IP numbers. Vulnerable to denial-of-service attacks.

No support for going through firewalls and proxies.

Low end users can not join meetings. No media processing is

provided.

Easy to use and understand.

(7)

H.323 Based Systems

There are many companies that provide H.323 based

videoconferencing systems. Polycom, FVC, etc.

Does not scale well.

H.235 defines the security mechanisms but most H.323 based systems

do not implement it yet.

H.323 based systems are not firewall friendly. It requires almost all

ports to be open.

Limited number of heterogeneous clients can be supported.

Not very easy to understand and develop services.

(8)

A B C

MC D E F

Audio Video

Multicast or unicast bus

A

B

D E

F

MCU

C

Audio Video

H.323 Centralized Multipoint Conferencing H.323 Decentralized

Multipoint Conferencing

(9)

VRVS

(Virtual Rooms Videoconferencing System)

Uses software reflectors to distribute audio/video streams.

Not open source. No details available.

(10)

GlobalMMCS Overview

Videoconferencing Tasks:

Audio/Video Distribution

Media processing

Meeting management

Meeting Management

Unit

NaradaBrokering Messaging Network

Media Processors Media Processing Unit

user

user _user _user

(11)

Evaluation of GlobalMMCS

Scalability:

Provides scalability by separating media

processing from media delivery.

Security:

NB provides all security services. It also takes

precautions against denial of service and replay attacks.

Traversing through firewalls, proxies and NAT

Supporting heterogeneous clients:

Since we provide a

scalable media processing framework and many transport

protocols, we can support a diverse set of end points.

(12)

Media Distribution Middleware

(NaradaBrokering)

Requirements for Media Delivery

High Bandwidth

Low latency

Tolerate Package Loss

NaradaBrokering

NB organizes brokers in a hierarchical cluster-based architecture.

NB supports dynamic broker and link additions/removals.

Messages are routed only to those brokers that have at least one

subscription

NB has a flexible transport mechanism

(13)

SSC-A SC-1 SC-2 SC-3 e g c 4 5 6 b f d a SSC-C SC-7 SC-8 SC-9 s u o q t r p

SSC-D SC-11_y z SC-10 w x v SSC-B SC-4 SC-5 SC-6 l n i j m k h

(14)

Incorporating Support for Audio/Video

Delivery in NaradaBrokering

Adding support for an unreliable transport protocol, UDP

Implementing a distributed topic number generation

mechanism

Designing a Unique ID Generation Mechanism

Designing a new compact event

Adding support for legacy RTP clients

(15)

Implementing Distributed Topic Number

Generation Mechanism

The requirements for topic number generation: spatial independence of a topic generator temporal independence of a topic generator Acceptable size

One topic number generator runs in every broker

20 bytes topic generator id guarantees spatial independence 220 = 1,048,576 topic number generators

44 bytes timestamp provides temporal independence 244_{=17592186044416 distinct timestamp values}

557 years with one millisecond resolution. UUID solves a similar problem with 16 bytes

This mechanism can also be used to generate unique ids.

20 bits 44 bits

Topic Number

(16)

Designing a New Event

In publish/subscribe messaging systems, messages tend to have many headers. A message in JMS API has at least 10 headers. These headers take around 200 bytes when they are serialized to transfer over the network.

A ULAW audio package for 20 ms has a size of 172 bytes and entails 64 kbps network bandwidth. Padding an extra 200 bytes of header to each audio

package results in the bandwidth requirement of 148 kbps. It is also more costly to serialize/de-serialize more headers. RTPEvent has four headers and 14 bytes long.

Event and Media headers are 1 byte each Topic Name is 8 bytes

Source info is 4 bytes.

Used to route messages

intelligenty in system Eliminates echo problem

Identifies the media type of the Event

Event Header

Media

Header Topic Name Source Info RTP Payload

Identifies Event as

RTPEvent RTP Payload

RTP Header

(17)

Supporting Legacy RTP Clients

RTPLinks receive raw RTP packages over UDP or Multicast from legacy systems, wrap them in RTPEvents and propagate them through the broker node. It also receives RTPEvents from the broker node and sent them as raw RTP packages to clients.

Each RTPLink starts two sockets: one for RTP and the other for RTCP. Similarly, it subscribes to two topics: one for RTP and the other for RTCP. Some RTP sessions might have more than one media stream, in that case, each stream might be published to a different topic.

(18)

Performance Tests of NaradaBrokering

The Characteristics of Audio and Video Streams

Quality Assessment of Media Delivery

Performance Tests for One Broker

(19)

Characteristics of Audio and Video Streams

Audio streams are composed of fixed size packages with regular intervals. We chose 64 kbps ULAW audio stream to be used in the tests:

One audio package is sent every 30ms. Each audio package is 252 bytes. There are 4100 packages in total, during 2 min.

Video codecs also encode frames periodically. However, each frame may have multiple video packages. Full picture update frames have much more packages. We chose H.263 video format, avrg. bandwidth 280kbps, for 2 min:

15 frames are encoded every second. One frame every 66ms.

1800 frames and 5610 packages in total. On avrg. 3.1 packages per frame. One full picture update every 60 frames or 4 seconds.

(20)

Quality Assessment of Media Delivery

There are three important factors: latency, jitter and package loss

ITU recommends that the mouth-to-ear latency of audio should be Less than 400ms for acceptable quality

Less than 300ms for good quality

Less than 150ms for excellent quality. The total latency is the combination of:

Processing at sender and receiver Transmission latency

Routing latency by the broker network We limit the routing latency to 100ms at most.

The packages that take more than 100ms are labeled as late arrivals. We limit the jitter caused by routing to 10ms

(21)

Performance Tests for One Broker

Single Meeting Tests

Single audio meeting tests

Single video meeting tests

Audio + Video meeting tests

Multiple Meeting Tests

Multiple audio meeting tests

Multiple video meeting tests

(22)

Single Meeting Tests

One transmitter and 12 measuring receivers. Other receivers are passive. Tests are conducted in a Linux cluster with 8 identical machines. These

machines had Double Intel Xeon 2.4GHz CPUs, 2GB of memory with Linux 2.4.22 kernel. All programs are written in Java. There is gigabit connection among the cluster nodes.

NB Broker

Machine 1

Passive Receivers

Passive Receivers Machine 2

Machine 3

Machine 4

Machine 8 Audio/Video

Transmitters

(23)

Single Audio Meeting Tests I

Number Of Clients L(1) (ms) L(N) (ms) L(av) (ms) J(av) (ms) LA(av) (%)

12 0.5 0.7 0.6 0.18 0

100 0.5 2.3 1.4 0.15 0

200 0.5 4.1 2.3 0.18 0

400 0.5 7.9 4.2 0.21 0

800 0.5 15.5 8 0.18 0

1200 0.5 22.6 11.6 0.22 0

1400 0.5 26.5 13.5 0.26 0

1500 3.3 32.3 17.8 0.44 0.25 1600 2260 2290 2275 1.2 100

0 5 10 15 20 25 30 35 40

0 500 1000 1500 2000

number of participants

(24)

Single Audio Meeting Tests II

The latency of first user is constant and does not depend on the number of users in a meeting

Each audio package is independent of others. The routing of each package is completed before the next one arrives. All audio packages in the audio stream takes almost the same amount of time to arrive to a client.

The broker saturates when the latency of the last user is more than 30ms. 1500 users can be supported in an audio meeting

Audio Pack age Latencies for the Las t Us e r in Single Audio M ee ting w ith 800 Participants

0 5 10 15 20 25 30 35 40

0 500 1000 1500 2000 2500 3000 3500 4000

Pack age Num be r

L

a

te

n

c

y

i

n

m

(25)

Broker saturation in single audio meeting

Latency values for the middle user in single audio meeting

with 1600 participants.

0 1000 2000 3000 4000 5000

0 1000 2000 3000 4000

Package Number

L

a

te

n

c

y

i

n

m

(26)

Single Video Meeting Tests I

Number Of Clients L(1) (ms) L(N) (ms) L(av) (ms) J(av) (ms) LA(N) (%)

12 1 1.3 1.2 0.44 0

100 3.1 5 4 2 0

200 6.3 10.2 8.3 4.7 0

300 10.2 16.2 13.2 7.8 0

400 13.4 21.2 17.3 10.1 0.75 500 18.2 28.5 23.4 13.2 3.0 600 22.6 34.5 28.6 15.5 5.1 700 29.8 43.7 36.8 18.1 8.4 800 45.6 61.6 53.6 21.3 17.6 900 93.7 111.7 102.7 23.8 40.8 1000 1599 1619 1609 27.8 99

0 20 40 60 80 100 120

0 500 1000 1500

number of participant

(27)

Single Video Meeting Tests II

Latency values for the last receiver in single video meeting with 400

participants.

Peaks correspond to full picture update frames.

One broker can support at most 400 participants because of late

arriving packages. Although the broker is saturated when there are

1000 participants.

The main reason for the late arriving packages are the full picture

updates.

0 20 40 60 80 100 120 140

L

a

te

n

c

y

i

n

m

(28)

Audio and Video Combined Meeting Tests

Each one affects the other.

Our initial tests showed that the impact of video meeting on the

performance of an audio meeting is significant. Therefore, we gave

priority to audio routing at the broker.

There are two queues at the broker: audio and non-audio. If an audio

package arrives, it is routed first as long as the routing of the currently

routed package is over.

When there are 600 participants, there is only 5ms difference.

Therefore, the impact of the video meeting is not significant on the

performance of the audio meeting

0 3 6 9 12

0 200 400 600 800 1000

num be r of pa rticipa nts

A v e ra g e l a te n c y i n m s

(29)

Comparison of single video meetings and

audio + video meetings

This test shows that the impact of an audio meeting on the

performance of a video meeting is not significant.

In audio and video combined meetings, the broker supports almost the

same number of participants as in the case of single video meetings.

The main reason for this is the better utilization of broker resources

when there are two concurrent meetings.

0 10 20 30 40 50 60

0 200 400 600 800 1000

A

v

rg

.

la

te

n

c

y

i

n

m

s

(30)

Multiple Video Meeting Tests

Total users # of Meet ings L(av) (ms) J(av) (ms) LA(av)

100 5 2.25 0.68 0

200 10 2.74 0.85 0

300 15 3.17 0.86 0

400 20 4.54 1.1 0

500 25 5.94 1.3 0

600 30 6.8 1.37 0

700 35 10.6 1.52 % 0.7

800 40 81.1 1.8 % 19

900 45 2787 3.3 % 98

0 20 40 60 80 100 120

0 200 400 600 800 1000

Total Number of Receivers

A v rg . L a te n c y i n m s

(31)

Latency values for each video package when

there are 30 meetings with 600 participants.

This graph shows that there are no peaks in latency values

for full picture update frames as it was the case in the

single video meeting case.

0 5 10 15 20 25 30 35 40

0 1000 2000 3000 4000 5000

package number

la

te

n

c

y

i

n

m

(32)

Summary of Single Broker Tests

1500 participants are supported in one audio meeting

400 participants are supported in one video meeting

Up to 400 audio participants and 400 video participants are

supported in audio + video meetings.

700 participants can be supported in 35 video meetings

each having 20 participants

1300 participants can be supported in 65 audio meetings

each having 20 participants

(33)

Performance Tests for Distributed Brokers

We have given priority to inter-broker package delivery over local

client deliveries.

This lets packages to travel many brokers with very little overhead. It

lets the broker network to scale.

It also eliminates cases where one overloaded broker severely affects

the performance of other brokers.

First Queue

Inter Broker

router

To Brokers

Second Queue

To local clients Thread 1

Thread 2 client

(34)

First User (ms) Mid User (ms) Last User (ms) Avr. (ms) Broker 1

400 users 15.8 20.2 24.5 20.2 Broker 2

(6 users) 16.1 16.1 16.2 16.1

First User (ms) Mid User (ms) Last User (ms) Avr. (ms) Broker 1

400 users 16.1 20.5 24.9 20.5 Broker 2

(6 users) 1.4 1.5 1.6 1.5

Test results with single and double queuing

(35)

Single Video Meeting Tests

for Distributed Brokers

There are equal number of participants in each broker.

We gather results from first and last user from each broker.

Machine 1

Broker 1 Broker 2

Broker 3 Broker 4

Video Transmitter

Video Receivers Video

Receivers Video

Receivers

Measuring Receivers

(36)

Latencies from 4 brokers

Broker1 and Broker2 have very similar latency values.

Broker3 and Broker4 have similar and slightly better latency values.

Going through multiple brokers does not introduce considerable overhead. Scalability of the system can be increased almost linearly by adding new brokers. 0 30 60 90 120 150 180

0 200 400 600 800 1000

Number of receivers

(37)

Multiple Meeting Tests

for Distributed Brokers

The same setting as the single video meeting tests. However, all

broker were running at cluster 2.

The behavior of the broker network is more complex when there are

multiple concurrent meetings compared to having a single meeting.

Having multiple meetings provide both opportunities and challenges.

Conducting multiple concurrent meetings on the broker network can

increase both the quality of the service provided and the number of

supported users as long as the size of these meetings and the

distribution of clients among brokers are managed appropriately.

(38)

Multiple Video Meeting Tests

4 brokers can support 48 meetings with 1920 users in total with excellent quality.

This number is higher than the single video meeting tests in which four brokers supported up to 1600 users.

When we repeated the same test with meeting size 20, 1400 participants can be supported. Number of Meetings Total users Broker1 (ms) Broker2 (ms) Broker3 (ms) Broker4 (ms)

40 1600 3.34 6.93 8.43 8.37

48 1920 3.93 8.46 14.62 10.59

60 2400 9.04 170.04 89.97 25.83

Number of Meetings Total users Broker1 (%) Broker2 (%) Broker3 (%) Broker3 (%)

40 1600 0.00 0.00 0.00 0.00

48 1920 0.12 0.29 0.50 0.50

60 2400 0.16 1.30 2.51 2.82

(39)

Wide-Area Media Delivery Tests

We tested with five distant sites:

Syracuse, NY,

Tallahassee, Florida, Cardiff, UK

Two sites at Bloomington, IN

We tested two cases:

(40)

Summary of Wide-Area Tests

Running brokers at distributed locations has many benefits:

Saves bandwidth, and eliminates bandwidth limitations.

Transferring smaller number of streams yields better transmission

services with smaller latency, jitter and loss rates.

Load is distributed to many brokers, more users can be served with

better quality services.

sender-to-receiver transmission latency can be reduced

considerably by running brokers at geographically distant

locations.

The networks that we used provided excellent services with very small

loss rates, latency and jitter values.

The network connections need to be checked for high quality. Cardiff

site was not even able to support 10 video streams (3Mbps), way

(41)

Meeting Management Architecture and

Services

There are three main components.

Meeting Management Unit starts/ends meetings, handles user joins and leaves. Media Processing Unit provides; audio mixing, video mixing and image

grabbing.

A unified framework is provided to distribute service providers and to manage the interactions among system components.

RLM Broker 2

RLM Broker 1

RLM Broker N

RTP Link Manager

Meeting Management Unit

Meeting Schedulers

Meeting Managers

Video Session Audio Session

NaradaBrokering Media and Content Distribution Network

MediaServer Manager

Media Processing Unit

Video Mixer Servers

Image Grabber Servers Audio Mixer

(42)

Messaging Among System Components

Although some messages are sent to a group of destinations, many

messages are destined to one target. Therefore, an efficient message

exchange mechanism should be designed.

We use reliable JMS messages to provide communications among

various components in the system.

This simplifies building a scalable solution, since messages can be

delivered to multiple destinations without explicit knowledge of the

publisher.

Messaging Semantics

Request/Response messaging

Group messaging

(43)

Topic Naming Conventions

Two types of topics are needed; group topics and unique component topics

All topic names start with a common root, GlobalMMCS.

Group topic names are constructed by adding the component name to the root GlobalMMCS/AudioSession

GlobalMMCS/AudioMixerServer

Unique component topic names are constructed by adding the unique ids: GlobalMMCS/AudioSession/<sessionID>

GlobalMMCS/AudioMixerServer/<serverID>

Sometimes a component communicates with many different components; in that case, there is one more layer to distinguish these communication channels

(44)

Service Distribution Framework

A unified framework to distribute many types of service providers

Addressing

: Each service provider and consumer is identified by a

unique topic name.

Service Discovery

: Dynamic discovery mechanism.

Inquiry

&

ServiceDescription

messages.

Service Selection:

the consumer selects the best service provider.

Service Execution:

the consumer executes the service by sending an

Request

message.

Broker Network Service Provider 3

Service Provider 2 Service Provider 1

Service Provider N Consumer 3

Consumer 2 Consumer 1

Consumer M

•Advantages:

• Fault tolerant • Scalable

(45)

Session Management

Audio and video sessions are managed separately.

AudioSession

objects manage audio sessions and

VideoSession

objects manage video sessions

MeetingManager

objects act as factories for session

objects. They initialize and end them.

AudioSession

and

VideoSession

objects provide session

management services to participants, such as user joins

and leaves. While handling these requests, they usually

(46)

Broker Network

user A

RLM Broker A

GlobalMMCS/AudioMixerServer/<serverID>

Audio Mixer Server

Audio Session

GlobalMMCS/AudioSession/<sessionID>/AudioMixerServer

RTP Link Manager

GlobalMMCS/AudioSession/Participant/<userID> GlobalMMCS/AudioSession/<sessionID>/Participant GlobalMMCS/RtpLinkManager/<brokerID>

GlobalMMCS/AudioSession/<sessionID>/RtpLinkManager

GlobalMMCS/AudioSession/<sessionID>/RtpEventMonitor

(47)

Audio Mixing & Performance Tests

D R PQ

Audio Mixer Stream 1

D R PQ

Stream 2 S E

Multiple Streams Speaker

D R PQ

Speaker 1

D R PQ

Speaker N

E To listeners

S E To Speaker 1

S E To Speaker N

D: Decoder To Multiple Streams Speaker Number Of mixers CPU usage % Quality 5 12 No loss 10 24 No loss 15 34 No loss Negl.

6 speakers in each mixer. Two of them were continually talking.

One more audio stream constructed with the mixed stream of all speakers. All streams were 64kbps ULAW.

(48)

JMS message paths for a VideoSession

Broker Network

user 1

RLM Broker A

GlobalMMCS/ImageGrabberServer/<serverID>

Image Grabber Server

Video Session

GlobalMMCS/VideoSession/<sessionID>/ ImageGrabberServer

RTP Link Manager

GlobalMMCS/VideoSession/Participant/<userID> GlobalMMCS/VideoSession/<sessionID>/Participant

GlobalMMCS/RtpLinkManager/<brokerID> GlobalMMCS/AudioSession/<sessionID>/RtpLinkManager

GlobalMMCS/VideoSession/<sessionID>/RtpEventMonitor

Video Mixer Server

(49)

Video Mixing & Performance Tests

Four video streams are mixed into one video stream.

Incoming video streams were 150 kbps H.261 stream.

Mixed video stream was H.263 with 18 fps. Linux machine with 1 GB memory and 1.8GHz Dual Intel Xeon CPU.

Only 3 video mixers are served.

(50)

(51)

Image Grabbing & Performance Tests

The purpose of image grabbing is to provide users with a meaningful

video stream list in a session.

Generated images can either be published on topics on the broker

network or can be saved to files.

An image is saved every 60sec to the disk in JPEG format.

The video stream was an H.261 stream with an av. bw of 150 kbps.

50 image grabbers can be supported on this machine.

The same machine as video mixing test.

Encoded

Image

Decoder

Video

Stream

Resizer

Encoder

Buffer

Looping_Thread

Number of IG

CPU usage %

10 15

20 35

30 50

(52)

Media Processing Service Distribution

MediaServers act as factories for service providers. They start/stop/advertise them. Each MediaServer is independent of others. New ones can be added dynamically . Service providers can either be started from command line when starting the

service container, or they can be started by using the MediaServerManager.

New services are assigned to least loaded media processing units.

NaradaBrokering Broker Network

JMS Messages

SP: ServiceProvider MediaServer

Manager 2

SP 1 SP 2

SP N

MediaServer K MediaServer

Manager M

MediaServer Manager 1

JMS Messages

SP 1 SP 2

SP N

MediaServer 1

SP 1 SP 2

SP N

(53)

Conclusion

Main Contributions:

Proposing a new architecture for scalable videoconferencing that

separates media distribution, media delivery and meeting management Investigating the issues related to using publish/subscribe systems for real-time audio/video delivery

Analyzing the performance and the scalability of NaradaBrokering broker network for the distribution of real-time audio and video streams in

videoconferencing sessions.

Implementing a meeting management and media processing service distribution mechanism based on publish/subscribe middleware.

Future Works

Media processing service distribution algorithms may be developed for large scale deployments

(54)

Publications

Ahmet Uyar, Wenjun Wu, Geoffrey Fox. “Service-Oriented Architecture for a Scalable Videoconferencing System”. Submitted to IEEE International

Conference on Pervasive Services 2005 (ICPS'05) 11-14 July 2005, Santorini, Greece.

A. Uyar, G. Fox. “Investigating the Performance of Audio/Video Service Architecture I: Single Broker”. To be presented at The International

Symposium on Collaborative Technologies and Systems. May 2005, Missouri, USA.

A. Uyar, G. Fox. “Investigating the Performance of Audio/Video Service Architecture II: Broker Network.” To be presented at The International

Symposium on Collaborative Technologies and Systems. May 2005, Missouri, USA.

Ahmet Uyar, Shrideep Pallickara, Geoffrey Fox. “Towards an Architecture for Audio/Video Conferencing in Distributed Brokering Systems”. The