Communication architecture

2.2.2. Communication architecture

The previous section has set the agenda for this thesis: support for social factors and expressiveness in large-scale CVEs. This section provides supporting consideration of network and software architectures for this type of system. It reviews and reflects on the approaches to network communication adopted in the systems considered in the previous section. In particular it considers the use of peer-to-peer networking which is adopted for both of the prototype systems described in this thesis and introduces and explores the use of network supported multicast communication which is a key ele- ment of part II. This section provides background material for the prototype systems and evaluations presented in this thesis. The aspects of communication architecture which are considered here are:

• “patterns” of communication and distribution, in particular use of direct inter-peer communication and the role of centralised servers and services; and

• use of network multicast facilities.

In each case the main issues and choices are described and compared, with examples from the multiuser virtual reality systems referred to in the previous section. This is summarised in table 4 on page 22.

Table 4: communication architectures of multiuser VR systems

System Peer communication? Servers? users in/out of system “core”? multicast in core

AVIARY yes yes: system in no WAVES via message

managers

yes: system in no

NPSNET yes no in yes

Paradise yes yes: system in yes Spline yes: in core yes: http and

system

in or out yes [Broll,

1997]

yes: in core yes: http and system

in or out yes RING yes: in core

(distributed server)

yes: system out: servers in core no DIVE yes yes: http and

system in yes Virtual Society yes: in core (distributed server) yes: http and system

out: servers in core yes NetEffect limited (one audio

call)

yes: http and system

out: servers in core no dVS yes yes: system in no

2.2.2. Communication architecture

Patterns of communication

The first issue considered is that of communication pattern, or the way in which com- munication is structured and organised within the system. A common distinction is between:

• peer communication, which takes place between comparable programs or proc- esses with similar or identical roles; and

• client-server communication, in which one particular process (the server) provides a service to one or more other processes (the clients).

Distributed systems are likely to include elements of both peer-to-peer and cli- ent-server communication. For example, some elements of any distributed system naturally lend themselves to implementation and presentation as a service. Examples include name services and traders (such as the ODP/ANSA trader [ITU-T, 1995]). However there is more diversity in realising communication between comparable processes, such as between participants in a CVE. The main options are direct inter-peer communication or indirect communication via a common server. The attractions of direct peer communication are: typically reduced latency; the central server cannot become a bottleneck; there are no issues of single versus distributed server; and it is relatively straight-forward to make use of multicast communication facilities between peers. Conversely, the attractions of communication via a common server are: tailored and lower-overhead communication with each client; simpler communication for the client processes; and simple inclusion of other services. Examples of CVEs which adopt client-server communication models are RING, Vir- tual Society and NetEffect. In the latter two cases this is directly motivated by the choice of a PC using a normal dialup modem connection as the principle delivery platform. The bandwidth limitations over the modem link are a critical consideration; this leads to a choice of a client-server architecture with tailored (e.g. compressed) communication between each client and the server(s). All three of these systems have a multi-server architecture in which a number of servers can cooperate to accommo- date a larger number of clients. In the case of NetEffect this is a simple federation of servers, each catering for one or more complete worlds. In the case of RING the serv- ers may act either as a fixed entry point to the core system, or may take responsibility for particular regions of the total virtual environment.

WAVES adopts a kind of hybrid architecture in which most communication is per- formed via general purpose message filtering processes which act as tailorable mes- sage routers.

The other systems all adopt a peer communication model, though with varying degrees of dependence on remaining server components. The Distributed Interactive Simulation [IEEE, I993] approach has traditionally avoided any form of server, rely- ing instead on extensive pre-configuration of systems. This is seen in the NPSNET and PARADISE systems. The remaining systems depend on some form of sys- tem-specific server, often in conjunction with standard HTTP servers. For example, AVIARY relies on a (potentially distributed) Environment Database (EDB) to main- tain world state and to perform collision-driven introductions between objects.

The approaches adopted by each system are shown in table 4 on page 22. A good combined approach is to employ peer communication within the “core” of the system and client-server communication at the “edges”. This is seen (in slightly different

2.2.2. Communication architecture

forms) in Spline, DIVE, the work of Broll, RING and in the distributed server of Vir- tual Society. In Spline, DIVE and the work of Broll the core system - where peer com- munication is employed - may include participants’ machines. In the cases of RING and Virtual Society the core system is restricted to the components of the distributed server. However in every case participants with restricted network access depend on client-server communication to participate. In the cases of DIVE and the work of Broll this compensates only for non-availability of local multicast facilities rather than for limited access bandwidth.

The work presented in this thesis has not focused on delivery over low bandwidth or non-multicast access networks and so peer-to-peer communication has been the approach adopted in both prototypes. As in most of these systems additional server components are employed as appropriate.

Use of multicasting

The second aspect of communication architecture which is considered in this chapter is the use of network multicasting facilities. Normal network communication is point-to-point, or “unicast”. Each message which is sent is received by (at most) a sin- gle remote machine to which it was explicitly directed. In the case of multicast com- munication a single message may be sent to an abstract “multicast group” and received by any number of other machines or processes which are notionally mem- bers of that group. The actual copying of messages is handled by the network, typi- cally through a combination of its physical characteristics (in the case of shared physical medium networks such as Ethernet) and by routers spread throughout the network. Figure 4 on page 25 illustrates the potential benefits of network supported multicasting when a sending machine (S) wishes to send the same message to four recipients. Note in particular the reduced number of packets on the WAN link and the shared LAN. Multicast communication has become available on UNIX operating sys- tem platforms in the past few years (and more recently on Microsoft’s Windows 95 and NT platforms) in the form of IP multicasting [Deering, 1989; Deering and Cheriton, 1990]. This is supported over wide area networks by the experimental MBone (Multicast back-bone) network [Macedonia and Brutzman, 1994]. Native multicast support is also becoming available on some commercial IP routers.

Multicast communication is extremely attractive for large-scale CVEs because it avoids carrying duplicate packets over the network. With unicast communication this would arise when, for example, one user was communicating with a number of other users over a common intermediate network (whether it be a shared LAN or an interna- tional link). With multicast communication only a single copy of the communicated data is sent over each common network and is copied as necessary to be distributed to the observers’ machines. The primary advantage of multicasting is to reduce the total bandwidth requirements on shared networks and links. The disadvantages of multi- casting are that: it requires support in the network which is not universally available at present; it requires some form of group management and group state in the network; and each group member receives the same messages (ignoring packet losses) even though they may have different detailed requirements. General networking issues such as achieving reliability, flow control and congestion detection and response are rather more difficult in the context of multicasting. It is also difficult to employ multi- casting in a client-server situation because each client typically has its own unique (though possibly overlapping) requirements.

2.2.2. Communication architecture

Of the systems which have been considered in this chapter the following employ mul- ticasting: NPSNET, PARADISE, Spline, the work of Broll, DIVE and the Virtual Society distributed server. Where such a distinction can be made this is confined to the core of the system. “Second class” client-server based users in Spline, DIVE and the work of Broll rely on normal unicast communication (typically using the TCP/IP protocol). This is summarised in table 4 on page 22.

In systems which employ multicasting the multicast groups are typically associated with the same elements which scope interaction or awareness, primarily cells or locales. This reduces the number of multicast groups which are required compared to the total number of objects in the virtual world and so limits the multicast-related state and management overhead in the network and the end-machines. This approach is to be expected in current systems since scoping of awareness is largely a pragmatic con- cern of limiting network and machine resource requirements, rather than actively sup- porting and structuring communication and collaboration. One of the challenges addressed in this thesis is the combination of a sympathetic and appropriate use of multicast communication with more active and expressive management of awareness. Of the two prototypes described in this thesis the first employs unicast communica- tion while the second employs multicast communication. In terms of implementation this was one of the principle developments between the two systems. This can be seen in chapters 6 and 10. The potential use of multicasting was also an important consid- eration in developing and applying the third party object extension to the spatial model of interaction which is presented in chapter 7.

Without multicast shared LAN WAN link R1 R2 S With multicast shared LAN WAN link R1 R2 S

S Sender _R Router Receiver Network Packet