Control and Coordination of Interactive Videoconferencing over Hybrid Networks
Ting-Chao Hou, Chorng-Horng Yang
†., Yun-Sun Chu, and Kim-Joan Chen
Department of Electrical Engineering National Chung Cheng University
160, San-Hsing, Ming-Hsiung Chiayi 621, Taiwan, R.O.C.
.† The author is now with the Wufeng Institute of Technology.
Abstract−−−−This paper describes the field trial experience on the control and coordination of an interactive videoconferencing system. The system employs ATM and Internet networks to realize the continuous presence and the conference control and management, respectively. Four control and management functions are designed to closely emulate interactive videoconferences. Unlike middleware approaches, we developed a service-specific, programmable control and management platform for multimedia applications. Based on the platform, a centralized architecture is employed to realize the conference control and management in a prototype system. Running the prototype system in Taiwan ATM/Internet networks, we observed diverse aspects of the system behavior. The control and coordination behavior naturally depends on the specified service and designated architecture and noticeably reveals the characteristics of the underlying Internet/ATM networks.
I.INTRODUCTION
The ITU-T (International Telecommunication Union− Telecommunication Standardization Sector) has developed many standards [1] for audiovisual services. With these standards, several videoconferencing systems have been developed [2]-[9]. The videoconferencing on N-ISDN employs the multipoint control unit (MCU) [10] for both the audio/video processing and the conference control. Using the MCU, the continuous presence [2] can be realized. Unlike the MCU approach, the Internet videoconferencing makes use of the Mbone [3] to distribute live audio/video of the conference. The audience must join the Mbone multicast group to receive live audio/video. Some management tools, e.g., session directory and whiteboard, support the audience to select the session and provide the feedback, respectively.
As the new computing and networking technology emerges, more sophisticated videoconferencing has been investigated. The videoconferencing includes the ATM-based video-conferencing [4]-[7] for supporting high-quality audio/video, the IN-based videoconferencing [8] for realizing the versatile service, the highly realistic videoconferencing [9] for achieving the virtual reality, etc. In [4], the middleware approaches (e.g., CORBA) are employed to realize an ATM-based videoconferencing application. In [5], a prototype application, ATM RendezView, can provide conferees with high-quality audio/video and demonstrate the continuous presence. In [6], a VideoMan is realized on basis of a service-specific control architecture, which can make
better use of network resources and provide more efficient services.
The above work explored various aspects of video-conferencing systems, with each one running on a designated network. Since several networks with diverse characteristics coexist currently, the videoconferencing over hybrid networks is an emerging issue. In this paper we address the experiment results of an ATM-based Multipoint Video-conferencing (AMV) system, which runs on ATM networks for high-quality audio/video distribution and on Internet for conference control and management. A prototype AMV system makes use of an existing product, SVA system [11], to realize the continuous presence and employs a service-specific, programmable control and management platform (CMP) [7] to implement the conference control and management. We conducted field trials for the prototype AMV system in Taiwan ATM/Internet networks and acquired some valuable experience. The rest of this paper is organized as follows. Section II presents the AMV system. Section III describes the system behavior. Section IV discusses the control and coordination. Section V summarizes this paper.
II.THE AMV SYSTEM
A. System Specification
In order to emulate interactive face-to-face conferences, we specified the required functions of the AMV system as follows. 1) Continuous Presence: In face-to-face conferences, each conferee can control at will the receipt and assimilation of multimedia information. This is the well-known continuous presence. 2) Conference Control: Rules that define how a conference starts/terminates as well as how the conferees join/leave the conference are interpreted as the conference control. 3) Floor Control: The floor concept is designed to fairly offer conferees the opportunities for speaking and to well conduct the proceeding of the conference. We define the floor control to realize the floor concept. 4) Status Report: In a face-to-face conference, each conferee naturally has his/her specific status information (e.g., state (present or absent), duty (chairperson, speaker, or listener), etc.), which can be easily recognized by other conferees. In a videoconference, the remote conferee’s status information can not be obtained easily. Thus a management
function, status report, is required for videoconferences. 5) Failure Recovery: A face-to-face conference can successfully proceed without failures because the audio/video and request/response can be distributed and transmitted reliably in a meeting room. However, the videoconferencing service may be disrupted due to the network congestion and/or failures. Thus, failure recovery is needed in video-conferences.
Obviously, the continuous presence is the key function. The other functions constitute the conference control and management. Fig. 1 shows the functional blocks of the AMV system. Referred to Fig. 1, the multimedia distribution platform (MDP) is responsible for distributing high-quality audio/video by making use of ATM channels. The control and management platform (CMP) [7] is a platform that controls and manages the MDP by making use of the ATM channels and management functionality. Both the CMP and MDP on the ATM network constitute a multimedia application development platform, which can provide basic control/management operations (CMOPs) to be invoked by the modules upon it. The modules above the CMP are the continuous presence and the four control and management functions (for short, CMFs). Moreover, two user interfaces are required to provide conferees with easy ways to access the above five functions.
B. Multimedia Application Development Platform
We employed an existing product, the SVA system [11], as the MDP and developed the CMP. The SVA system contains two types of hardware devices: Audio Video Adapter (AVA), which compresses the analog audio/video signals into Motion-JPEG stream, and ATM TV (ATV), which decom-presses the Motion-JPEG stream back to analog audio/video signals. The software in SVA includes AVA/ATV managers, traders, and applications. AVA/ATV managers can control AVA’s/ATV’s operations and the trader provides the naming service of the managers for applications. With the trader’s naming service, the application can specify the parameters (e.g., frame rate) of an audio/video stream by controlling the source AVA manager, and create, patch together, and display video/audio streams by controlling source AVA and sink ATV managers. Thus, the SVA system can be used to realize the continuous presence.
In order to control and manage the MDP easily, the CMP bundles up tedious SVA instructions and makes use of the ATM control/management functionality to construct eight CMOPs. These CMOPs are as follows. 1) CAVS: Create a point-to-point/point-to-multipoint Audio/Video Stream. 2)
TAVS: Terminate an Audio/Video Stream. 3) XAS: miX Audio Streams. 4) XVS: patch (miX) Video Streams. 5) MSS: Monitor the State of a Stream. 6) RFS: Restore the Failed Stream. 7) MSDM: Monitor the State of multimedia Device Manager. 8) RFDM: Recover the Failed multimedia Device Manager. Further details of the CMP are referred to [7].
C. Architecture
In a distributed architecture, the software modules that realize the CMP and the four CMFs are running at different computers, so that network communications are required to coordinate these software modules. Thus, the synchronization of the data for control and management may become a critical issue. On the other hand, with the centralized approach, the software modules are running at a computer that is referred to as the control and management server (for short, CM server). Instead of network communications, only IPC (inter-process communication) is required and thus the coordination and synchronization can be dealt with more easily. So we adopted the centralized approach. Besides, a web-based interface that provides all conferees with an easy way to control and manage the videoconference is implemented. The architecture consisting of browsers and a web server is employed to realize the interface (see Fig. 2).
D. Control and Management Server
The CM server (see Fig. 2) consists of four main modules: operation manager, fault manager, dispatch, and CGI. A shared memory is employed for communications among the four modules. In order to synchronize the data in the shared memory, four semaphores are used to control the access to the shared memory. Besides, several tables and a floor queue are designed for the operation manager and the fault manager. We briefly explain how the CM server works as follows.
1) CGI and Web-based Interface: Users can initiate a command by clicking the command icon. The command is sent from the browser to the web server and then to the CGI that activates the corresponding CMF in the operation manager. After executing the CMF, the operation manager sends back the response to the browser through the CGI and the web server.
2) Operation Manager: The operation manager consists of four CMFs as well as the GCMI (generic control and management interface), coordinator, command processor, and state manager. We only explain how the conference/floor controls work as follows:
1) The conference control is responsible for the management of the conferee’s status that is recorded in
Control and Management Platform ATM Network Control Conf-Ctrl. Floor-Ctrl. Management S-Report F-Recovery Interface Audio/Video Continuous Presence Interface Multimedia Distribution Platform
Multimedia Application Development Platform
the user table. An entry in the user table is also associated with the multimedia devices and the device managers that are kept in the device table and the device manager table, respectively. Besides, the conference control also invokes the corresponding CMOPs through GCMI to accomplish the conferee’s interaction (e.g., join).
2) The floor control can receive conferees’ floor requests from the web-based interface and store them in a floor queue. Then, the floor control selects a floor request to serve for a predefined floor holding time. Many disciplines can be used to select the floor request, such as first come first serve, etc. The floor control then achieves the interaction, i.e., floor transfer, by invoking CMOPs to rearrange the audio/video distribution. 3) Dispatch: The dispatch consists of a channel manager (C-Mgr) and a number of ATM channels (Cs) for control and management. The C-Mgr is responsible for the setup and teardown of the ATM channels. Besides, the C-Mgr feeds instructions to the channels as well as collects the information from the channels. Such instructions and information are stored in the shared memory. (Note that we do not discuss the fault manager in this paper.)
III. SYSTEM BEHAVIOR
A. Experiment Results
We conducted field trials for the prototype AMV system in Taiwan ATM/Internet networks. From the field trials, we noticed some control and coordination issues.
1) Field Trials in the LAN Environment: The AMV system is demonstrated to provide the expected services (i.e., continuous presence and conference control/management) for conferees.
2) Field Trials in the WAN Environment: The AMV system can support up to nine conferees to participate in the video-conference. Multicast of audio and video information runs in the ATM WAN successfully and the CMFs provide the expected conference/floor controls and status report
services. However, we also observe the following four conferencing behavior and/or system characteristics: ER1) The outcome of the floor competition ideally should depend on both the floor requests that conferees initiate and the discipline that the floor control adopts. During the field trails, we found that the floor control may be biased against some conferees’ floor requests. ER2) The response time through the web-based interface can vary significantly, so that conferees may generate lots of repetitive interaction requests. ER3) The time required for synchronizing the audio/video streams depends on the distances between stream sources and sinks, total number of streams to be mixed or composed, and the traffic and topology of the ATM network. ER4) When conferees highly interact with one another, the AMV system may be overloaded and fails to generate responses for a long time. Conferees therefore initiate a large number of new or duplicate requests that may further make the system unstable or, more seriously, uncontrollable. Then, the service disruption occurs.
B. Control and Coordination Behavior Model
To discuss the experiment results, we make use of a model to capture the control and coordination behavior and distinguish the control and coordination abstractions. We explain the model and abstractions as follows: 1) Conferee Interaction: Referred to Fig. 3, the conferees closely interact by sending requests (e.g., join) to the CMFs through Internet. 2) CMOP Coordination: To achieve a conferee interaction, lots of CMOPs (e.g., for join, the CMOPs are CAVS, MSS,
MSDM, etc.) are invoked. The invocations of these CMOPs must be coordinated in order to preserve temporal relations (e.g., invoke CAVS first and then MSS and MSDM). 3) Control Flow Creation: To execute a CMOP, one or many control flows have to be created in order. For example, the CAVS
may involve at least two control flows: the first,
CtrlSrcAVA, and the second, CtrlSinkATV. 4) Control
Message Dispatch: A control flow is a series of consecutive control messages that are unicasted (multicasted) through the ATM network to the remote device manager(s) (for short, D-Mgr). GCMI Command Processor Conf-Cntrl Flor-Cntrl StateReport FailRecover
Coordinator ManagerState Operation Manager Restorator Monitor C-Mgr Dispatch CGI user D/D-Mgr d a c F -que ue x b a new serve circuit process D-mgr ATM cntrl/mangt MDP D-Mgr ... Fault Manager s s s shared memory Internet
Browser Browser...Browser
web server s C C D-Mgr D-Mgr CM server
Fig. 2. Software modules of the CM server.
CM O P C oor din ati on C ontr o l F low C re a tion Internet ATM Mgr D Mgr D Mgr D Control Messages Dispatch Dispatch join m1 m2 Command Processor m2 Network Congestion video source video sink-1 video sink-2 Usr-X Usr-Y Usr-Z Coordinator Operation Manager CA VS MS DM MS S m2 Browser Browser Browser m1 CMFs
IV. CONTROL AND COORDINATION
A. Formulation
The videoconference can be regarded as a series of interactions. During each interaction, the AMV system involves collecting interaction requests, processing the requests, controlling/coordinating the system, and providing audio/video for conferees. To facilitate the investigation of the control and coordination behavior, we made some assumptions as follows. 1) For each interaction, the floor control always results in the floor transfer and the speaker completely uses up the predefined floor holding time in addressing. 2) The join or leave request received by the CM server during an interaction is processed in the next interaction. The following notations and terminology are used in the discussion.
IVS: interactive videoconferencing service. i
I : i-th interaction of theIVS.
θ: predefined interval for floor competition. i
P: interval of processing interaction request in Ii. i
C: interval of controlling the AMV system in Ii. i
f : video freeze interval of Ii. i
S : interval of providing audio/video services in Ii. f
T : predefined floor holding time. Χ: interactivity of the IVS.
Referred to Fig. 4, the IVS consists of a series of interactions:IVS=i0,I1,...,In,in+1, where i0 and in+1 are the begin and end operations, respectively. The interval of each interaction is divided into four parts: θ, Pi, Ci, and
i
S . During the first interval (θ), conferees send the floor requests from the browsers to the CMFs through Internet (i.e., conferee interaction abstraction). During the second interval (Pi), the CMFs process the interaction requests and then generate the corresponding control and coordination scenario (i.e., internal processing of the CMFs). During the third interval (Ci), the CM server controls the AMV system according to the control and coordination scenario to rearrange the audio/video distribution (i.e., CMOPs coordination, control flow creation, and control message dispatch abstractions). Then the service is available during the service provision interval (Si). We define the video freeze interval fi of Ii as the interval from the end of the previous service provision interval to the start of the current service provision interval, i.e., fi=θ+Pi+Ci . In this interval, the audio/video information is out-of-date or meaningless to the current interaction. Then we can define the interactivity, Χ, as follows.
∑
= + = n i i i i S f S n X 1 1 , where Si =Tf (assumption 1). Due to the constant service provision interval, the shorterthe video freeze interval is, the larger is the interactivity. To achieve a high interactivity, we would like to shorten the video freeze interval. In other words, cut the floor competition interval if possible and accomplish the interaction request processing and the control and coordination quickly. However, if we shorten the video freeze interval improperly, control and coordination issues (e.g., ER1~ER4) may arise. We discuss these issues as follows.
B. Discussion
Referred to Fig. 4, the video freeze interval (say fi) consists of three parts: θ, Pi, and Ci. We assume that the CMFs spend a constant time in the interaction request processing (i.e.,Pi= p,∀i). This assumption is reasonable because the CM server spends less time in Pi than in θ or
i
C . (Note that network communications are involved during the θ and Ci phases.) ER1 arises mostly from the nondeter-ministic and the asynchronous characteristics of Internet. As shown in Fig. 4-(a), four conferees, A, B, C, and D, initiate his/her floor request at the time ta,tb,tc,and td, respectively. Each of the requests traverses through the browser and Internet to the floor control. For simplicity, we suppose that the browser spends a constant time b in processing the request. However, a request initiated by a conferee (say A) spends the time, dA+∆dA [12], in traversing through Internet to the floor control. The dA is the minimum delay from conferee A’s browser to the floor control; the ∆dA is the fluctuation part of that delay. Because Internet is a collection of various networks connected together, a longer route may have larger d and
d
∆ . Though conferee A initiates a floor request before conferee B (ta<tb), B’s request may arrive at the floor
control before A’s request due to
(
b a)
B B A
A d d d t t
d +∆ > +∆ + − . So the floor control is biased against the floor request that has larger d and/or ∆d.
On the impact of θ value, a smaller or improper θ (sayθ′) may forbid some conferee(s) competing for the
IVS
...
fi-1 si-1 fi si fi+1 si+1...
Tf Tf Tf θ pi ci A B C D θ ' θ "...
CMOP δ tc b du ∆du tb td ta δ δ(a) Interaction requests transmission. (b) CMOPs execution Fig. 4. Video freeze interval.
current floor. Fig. 4-(a) shows that conferee C’s request has a very large dC and conferee D’s request has a large ∆dD. These two requests can not reach the CMFs in θ′, so the CMF processes these requests in the next interaction. On the other hand, a large θ (say θ′′) can alleviate the above prohibition but makes the video freeze interval larger and reduces the interactivity. When the AMV system runs in a WAN environment, not only the value of d but also the value of ∆d can fluctuate in a wide range. So setting a proper value for θ to meet the interactivity requirement and resolve the prohibition is difficult. With an improper θ, the CMF may postpone processing the requests for an interaction. Therefore, both the variation in the values of d and ∆d and the postponement of the interaction request can result in the ER2.
During the control and coordination interval (Ci), the CM server executes the CMOPs to rearrange the audio/video distribution. Executing a CMOP involves the control flow creation and the control message dispatch in the ATM network. A research [13] in the ATM traffic and congestion control shows that the congestion caused by the highly bursty traffic (e.g., videoconferencing) arises and dissipates quickly and, moreover, the variability in loss rate is extreme. This can explain why the time for completing a CMOP can vary greatly. If the CM server sequentially executes CMOPs (or, in protocol vocabulary, stop-and-wait), the Ci interval will be very long and the interactivity will be low. Thus we adopted a pipelining approach for executing CMOPs as shown in Fig. 4-(b). The interval between executing two CMOPs is δ . By adjusting the δ value, different interactivity is obtained. Though small δ value implies high interactivity, the system may have a hazard of violating the CMOP ordering (i.e., finishing a CMOP before its prior CMOP) that may make the system unstable or, more seriously, uncontrollable. In an ATM WAN, the execution time of the CMOP can fluctuate widely due to the unpredictable congestion and the loss rate as mentioned above. So, it requires much more time to synchronize multiple video streams (ER3). Unfortunately, the δ value is sensitive to both the interactivity and the stability. When we select a δ value that can not accommodate the unpredictable ATM traffic and can not achieve the expected interactivity, the system may run out of control (ER4).
V.SUMMARY
A prototype ATM-based multipoint videoconferencing (AMV) system employs Internet/ATM for realizing the conference control/management and the continuous presence, respectively. During the field trials for the prototype AMV system in Taiwan ATM/Internet networks, we observed certain system behavior. We built a model and formalized the control and coordination behavior. The control and coordination involves conferee interaction, CMOP
coordination, control flow creation, and control message dispatch. Firstly, transferring the conferee’s interaction requests through the nondeterministic and asynchronous Internet may disturb the order of the requests, resulting in several problems such as fairness of floor control and fluctuation of response time. Secondly, executing CMOPs in a pipelined way can improve the interactivity but the δ value is sensitive to the system stability. Thirdly, an improper
δ value that does not accommodate the ATM traffic variation and can not support the expected interactivity may make the system uncontrollable. With the above experience and abstraction, this study has pointed out that the next generation application (e.g., videoconferencing) involves complex group communications. Existing multicast protocols may fail to meet the stringent QoS requirements of the multimedia information and preserve complicated ordering/ timing relations of the control messages. So, advanced group communication protocols should be explored to resolve these difficulties.
REFERENCES
[1] S. Okubo, et al., “ITU-T standardization of audiovisual communication systems in ATM and LAN environments,” IEEE J. Select. Areas Commun., vol. 15, pp. 965-981, Aug. 1997.
[2] W. J. Clark, “Multipoint multimedia conferencing,” IEEE Commun. Mag., pp. 44-50, May 1992.
[3] M. R. Macedonia and D. P. Brutzman, “Mbone provides audio and video across the Internet,” Computer, Apr. 1994.
[4] W. S. Choe, T. J. Geok, W. W. Guo, and T. S. Woon, “ATM-based multi-party conferencing system,” in Proc. IEEE GLOBECOM '95, pp. 592-6, Nov. 1995.
[5] K. Smith and R. Pretty, “ATM RendezView: multipoint conferencing on ATM,” in Proc. IEEE Int. Conf. Multimedia Computing and Systems, Jun. 1997.
[6] J. E. van der Merwe and I. M. Leslie, “Service-specific control architecture for ATM,” IEEE J. Select. Areas Commun., vol. 16, pp. 424-436, Apr. 1998.
[7] C.-H. Yang, T.-C. Hou, C.-C. Wen, H.-J. Yang, and K.-J. Chen, “Control and management architecture for multipoint video-conferencing over ATM networks,” in Proc. Sixth IFIP/IEEE Int. Symp. Integrated Network Management (IM ’99), 1999, pp. 887-900. [8] D. Faouzi, “Multimedia and multiparty control: the intelligent network
solution,” in Proc. Int. Conf. Commun. Technology (ICCT ’96), May 1996.
[9] S. Weik, J. Wingbermuhle, and W. Niem, “Automatic creation of flexible antropomorphic models for 3D videoconferencing,” in Proc. Int. Conf. Computer Graphics, June 1998.
[10] M. H. Willebeek-LeMair, D. D. Kandlur, and Z.-Y. Shae, “On multi-point control units for videoconferencing,” in Proc. 19th Conf. Local Computer Networks, Oct. 1994.
[11] Nemesys Research Limited, “SVA 2.0 user manual,” April 1996. [12] Q. Li and D. Mills, “On the long-range dependence of packet
round-trip delays in Internet,” in Proc. IEEE ICC ’98, June 1998. [13] Y. Jou and A. Nilsson, “ATM traffic measurement and congestion
control issues from VISTANET testbed,” in Proc. IEEE GLOBECOM ’95, Nov. 1995.
