Building Scalable and High Efficient Java Multimedia Collaboration

Building Scalable and High Efficient

Java Multimedia Collaboration

Wenjun Wu, Tao Huang, Geoffrey Fox

Global-MMCS & XGSP

n XGSP

• A common AV signaling protocol

• A core conference control mechanism • Collaboration Web-Services

n Global Multimedia Collaboration System

(Global-MMCS)

• to support scalable web-service based interoperable

collaborations

• integrates various services including videoconferencing,

instant messaging and streaming, and supports multiple

Global-MMCS & XGSP

SIP H323 Access_Grid Native_XGSP Admire

Gateways convert to uniform XGSP Messaging

High Performance (RTP and XML/SOAP and ..

Media Servers Filters Session Server XGSP-based Control NaradaBrokerin g All Messaging

Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing

NB Scales a distributed

We Services

Java Media Framework

n

Java Media Framework (JMF)

• A unified framework for managing the acquisition,

processing, and delivery of time-based media data

• A common cross-platform Java API for accessing underlying

media frameworks.

JMF for multimedia collaboration

n JMF is the key building block for the implementation

of Global-MMCS

For both client-side and server-side

Building multimedia services such as video, audio mixing, snapshot generation

Implementing Global-MMCS AVportlet, a native XGSP audiovisual client

n Performance Issues in JMF

Multimedia Programming Frameworks

Windows Core Audio Sequence Grabber Mac OS ALSA Video-for-Linux Linux Audio Video Capture

Portability vs Performance :

Platform-dependent multimedia devices and codec management

Multimedia Programming Frameworks

n the similar multimedia filter pipeline architecture

n Filter & Pipe design pattern

cascaded filter chain

n Support high efficient media processing

• High-volume multimedia data is typically generated

Multimedia Programming Frameworks

n Tcl/Tk based

CMT [7], VIC [8], VuSystem [9]

low-overhead control functionality implemented in a scripting language like Tcl

performance-critical data handling implemented in a compiled language like C/C++

n Windows DirectShow

C++ COM API

only enables Windows applications for multimedia programming

Access Grid

Access Grid : a large scale

audio/videoconference based on a multicast network

n provides the group-to-group

collaborations among 150

nodes connected to Internet 2 world wide.

n Use improved MBONE

audiovisual tools VIC and RAT

n Depends upon high-speed

Issues for JMF in collaboration

n Very good cross-platform framework

n No optimization for conferencing application

Compared to fully performance tuned Tcl/Tk package ( Access Grid )

Enhanced Java Media Framework

n Improving JMF performance by exposing the

interfaces of the JMF filters and rewriting a faster one

n Adding the state-of-art codecs in JMF and screen

capture

n Extending the JMF to the Mac-OS platform to extend

Video Rendering

n JMF filter pipeline is composed of filter Modules and

linking Connectors

n streaming protocols between a OutputConnector and

InputConnector: Safe protocol

Push protocol

n JMF render implementation:

safe protocol , blit the whole video frame

n Our optimization approach:

Video Capturing

n a video data source to abstract the real capture

services which usually have different APIs in modern operating systems.

n added Mac video capturing into JMF package

n Added the desktop capture as a video data source

Add New Codec

n JMF 2.0 package only supports H.261 decoder, H263

Global-MMCS AVPortlet

n GlobalMMCS AVPortlet, to fully make use of the

services provided by Global-MMCS. The tool which can run on multiple desktop platforms such as

windows, Linux and Mac OSX, integrates audio and video collaboration together

n Extending JMF RTP Transport over Publish/Subscribe

Overlay

Performance Evaluation

n Compare the video rendering performance between JMF and

VIC

Testing video streams are captured from the same desktop of the sending machine, and multicasted to the receiver machine

which runs Access Grid VIC, two JMF clients based on Sun’s JMF implementation and our improved version.

n two test scenarios with different source streams

• the CIF-size still image of the desktop area is sent to the clients. Each

stream is encoded in H.261, and has average bandwidth 20kbps.

• the CIF-size video sequence from a 30-second movie with a lot of motions

Performance Evaluation