Research Issues

In this dissertation we investigate the issue of developing a fault tolerant and service oriented streaming architecture based on messaging systems that integrates videoconferencing, streaming and annotation systems into an interoperable collaboration environment. The research issues in this thesis can be divided into four categories: session management, metadata management, multiple stream support and investigation of application scenarios devised to demonstrate the framework explained in this dissertation.

management of sessions and streams. Multiple stream level issues are related to data level support for fault tolerant streaming with quality-of-service.

1.2.1

Session Management

Videoconferencing systems have their own conference control framework which makes them accept only clients that use their protocols. In such a system the environment is homogenous, that is only one type of client exists in the videoconferencing session. Even when the systems define some gateways for other client types, such as SIP-H.323 gateway [37, 38] to enable SIP clients talk to H.323 terminals, the architecture makes it difficult to extend it to other types of clients and also these gateways are limited to only a handful of client types. This is because the protocol that they define either does not allow them to extend the architecture to cover heterogeneous client types or makes it too difficult to integrate with other systems. So we propose XML Based General Session Protocol (XGSP) for audio and video sessions to overcome these difficulties in achieving an integrated videoconferencing system. XGSP allows the system to be extended easily to different client types. We investigated how this is achieved to handle H.323 terminals, SIP clients and as well as MBONE clients.

We then propose a streaming gateway to investigate how flexible this protocol is in integrating videoconferencing systems and streaming systems which are two different technologies in multimedia streaming and collaboration. We demonstrate this for RealMedia [39-41] streams and also investigate other client types such as cellular phone clients that can be included in the system.

We also extend the XGSP framework to include RTSP functionality to support archiving and replay of streams. We propose Archived Streaming Service which provides metadata support and messaging middleware topic management for archiving streams and replaying archived streams.

1.2.2

Metadata Description and Management

We divide metadata into two categories; static metadata and dynamic metadata. Static metadata is the metadata that does not change during the lifetime of the session while dynamic metadata changes. We investigate session and stream metadata from these two categories.

We describe all metadata in XML format and store them in WS-Context Service [42] which provides a distributed fault tolerant metadata repository that can be shared among services and clients in the system. We investigate the impact of this on the design and architecture of the streaming service.

1.2.3

Multiple Stream Support

Since we will use messaging middleware, particularly NaradaBrokering publish/subscribe system, we propose developing new services and extending some existing ones in order to provide archive and replay streaming functionality within NaradaBrokering system.

We transport data whether it is audio/video or text or any binary data (i.e. image data) with the NaradaBrokering native event called “NB Event”. The data is encapsulated inside the event and published to a topic. At this stage, the only parameter needed to

provide streaming functionality for the events published to a topic is the timestamp information so that we can preserve the ordering and timespacing between the events. This lets us provide functionalities such as seeking within a stream, playing a stream at its original rate etc. The services that we propose are as follows:

Time Service: Unsynchronized clocks in a distributed environment will result in

inconsistency in timestamping events generated from different sources. In order to avoid this inconsistency in archiving and replay services, we propose using the time service within messaging environment which is provided to entities in the messaging system so that entities can timestamp the events. We will investigate the behavior of local clocks on Linux and Solaris machines.

Jitter Reduction Services: Jitter is considered to be one of the most important

quality of service measurements within audio/video collaborative systems. As a part of these services we propose a Buffering Service and Time Differential Service. These two services require the use of the Time Service mentioned above. We will demonstrate how Jitter Reduction Service using the following services is effective in reducing jitter in an intercontinental environment which we set up between Cardiff, UK and Bloomington, IN, USA. We investigate the delay introduced by this service to the stream.

1. Buffering Service: The goal of this service is to order events that clients

receive. Unordered events mean that the original order has changed. Especially if the publisher and the subscriber are located at a large distance from each other it is possible that some of the later events reach the subscriber earlier than the former.

2. Time Differential Service: The ordering of events is not sufficient to reduce jitter. If the timespacing of events is not preserved this will also increase jitter. We propose Time Differential Service that timespaces events before releasing them to the subscriber. This service is another means to preserve the rate at which the stream is published.

We also propose Generic Streaming Framework which can archive and replay any event type within the messaging environment. This framework also provides guidelines on how to archive and replay raw RTP packets as well as major RTSP functionalities (play, pause, rewind and forward).

We also investigate how we can synchronize streams using the services proposed in this research.

We also propose a replication service by introducing a replication scheme for messaging systems that can store events in redundant repositories to improve the fault tolerance of the system in the presence of repository failures.

1.2.4

Application Scenarios

We investigate application scenarios from different collaboration systems. From videoconferencing systems, we will archive and replay GlobalMMCS [43-49] sessions. GlobalMMCS sessions contain audio and video streams. We will also archive and replay sessions from Anabas eLearning and Collaboration [50] system. Anabas is a web based collaboration system which provides many different tools such of shared display, text messaging and audio conferencing. These two collaboration systems demonstrate different characteristics, since in GlobalMMCS sessions the number of streams (hence

topics) changes as clients join or leave sessions, in Anabas there is a fixed number of topics.

Another important application area is annotation systems. We initiate an annotation system named “eSports Collaborative Multimedia Streaming and Annotation System” based on the streaming architecture proposed in this dissertation. In eSports, we will replay real-time live A/V streams and annotate them. We shall also replay annotated streams with the annotations.