The complete E-training system is composed of the networked simulators and a dedicated server, as shown in Fig. 8-1. As described in the previous chapters, a key factor to simulate realistic obstetric ultrasound scan is that the simulator should be able to mimic freehand scan on the abdominal surface of a pregnant woman. Accordingly, all networked simulators should be able, on their own screens, to synchronously display the same movement of the virtual transducer on the virtual torso and display the same 2D ultrasound image. In other words, all networked simulators except one work as passive monitors, displaying the same 2D ultrasound image, virtual transducer etc., as on the active simulator, on which a user performs ultrasound scan. The passive and active simulators are referred to as the observer simulator and the operator simulator, respectively. To achieve this in a traditional design, an E-training system has to directly transmit the 2D images. Although real-time video streaming is technically feasible over high speed internets, such as broadband networks or 4G mobile networks in the United States or other developed countries, real-time video streaming over 2G/3G mobile or low speed networks is still a challenge. Such limited speed networks are often encountered in developing countries.
An alternative to resolving this problem is to only transmit the sham transducer’s position and orientation (the five degree of freedom data described in Chapter 3), so that the data transmission only requires a very low bit rate. This requires that all participants must have the same image volume loaded, which is ensured through software commands from the instructor.
140
Fig. 8-1. Workflow of the ultrasound training simulators in synchronous mode.
The E-training system was implemented with a client-server architecture for three reasons. First, the instructor simulator has administrative rights over all other simulators in order to manage the training mode and specifically assign a simulator to be the operator simulator. The client-server architecture is appropriate for processing an incoming connection request based on the sender’s identity (an instructor or a learner). Second, given that routers or gateways have been widely used in modern networks, the E- training system needs the server having a public IP to establish the communication between the simulators with and simulators without public network IPs. This usually requires a special technique, called Network Address Translation (NAT), to find the public IPs and port numbers of all participating simulators in private networks. Using a client-server architecture makes implementation of the NAT easier in the case of a simulator operating in a mobile or private network [117]. Finally, since only a limited number of simulator users (we assumed less than 10 in a typical scenario) participates the training at any given time, a client-server architecture is feasible to handle the communication among these networked simulators.
In the current design, the dedicated server must have a public IP address so that the simulator is able to send a connection request to the server based on its IP address. In addition to the connection establishment, the server also handles clients (or simulators)
141
management and relay of tracking data. The role of a simulator, either as an operator or observer, is determined by the instructor and thus must be dynamically changeable. In the synchronous mode, there is only one operator simulator at any time, broadcasting the transducer’s tracking data to other observer simulators. In our design, the instructor simulator and learner simulators share the same software design except that the instructor simulator has, as mentioned, administrative rights to manage the system.
8.1.1 Communication establishment
A simple, custom protocol has been developed for the E-training system to establish the communication between the server and clients. Specifically, the protocol is utilized to establish and monitor communication channels and to transmit text data through the TCP protocol. After communication to the server has been established, the sham transducer’s tracking data are transmitted via the UDP protocol, chosen because the UDP is able to operate with a much shorter delay than the TCP protocol. Each communication packet includes a header and a payload. The header portion defines packet type, as described below.
• Greeting: Designed for connection acknowledgement. Once the connection is
successfully established, the server and the client send a greeting packet to each other. • Update Client: When a new client joins or an existing client leaves, the server
broadcasts this packet that contains an updated client-list to all networked simulators. • Update Operator: When the instructor designates a new operator simulator, the server
broadcasts this packet to inform all networked simulators of the change of the operator simulator.
• Update Training Mode: The instructor configures the training as the synchronous or
asynchronous modes. The server broadcasts this packet to inform all networked simulators of the change of the training mode.
• Message: Designed for client communication by text.
142
Given that routers or gateways may exist in the network, a simulator (client) must establish a communication channel to the server before the simulator can begin sending or receiving tracking data. Therefore, a UDP hole punching approach was implemented for the E-training system. The process of the UDP hole punching is shown in Fig. 8-2 and described in the following paragraph.
Fig. 8-2. Server connection establishment.
The server first opens a dedicated TCP port for listening to incoming connection requests. The client then is able to initiate the connection to the server, whose public IP address is denoted A. Once the connection is successfully established, the client sends a greeting message that contains the client user name to the server. Then the server: 1) retrieves the public IP address (denoted B) of the client and its port number (denoted N); 2) combines them with the client user name to generate a unique client ID; 3) sends a
143
greeting message that contains the IP address and port information (B:N) back to the client; and 4) binds a UDP socket to port N. After receiving the greeting message from the server, the client sends a UDP packet to the address of A:N to make itself visible to the server in the case that the simulator runs in a private network. Finally, the server updates its client list and broadcasts the list to all networked simulators.
8.1.2 Data transmission
After the communication channel has been established, the operator simulator now can send the transducer’s tracking data to the server through the “punched” UDP port. The server then relays the tracking data to all observer simulators by searching a client table, which will be updated when a client joins or leaves the session. At the client side, a first-in-first-out buffer is used to queue the incoming tracking data so that the observer simulator is able to smoothly render 2D images. If the buffer is full when new data arrive, the simulator will decimate the tracking data in the buffer to make the buffer up-to-date. If the buffer is empty, the simulator will use the immediate past tracking data to render 2D images.
In addition to the tracking data, the system also establishes text channels among all clients by using the “Message” packet. Given that texts do not need to be synchronized, the system transmits them through the TCP protocol.
8.1.3 Management of the operator simulator
In the synchronous mode, only one operator simulator is performing the scan while all the other simulators are observers. The assignment of the operator simulator status is managed by the instructor who can dynamically change the role of any simulator from an observer to an operator or vice versa by sending an “update operator” packet to the server. A string variable at the server side stores the ID of the operator simulator. Then, the server informs all networked simulators of the change of the operator status. If the instructor does not appoint any simulator to be the operator simulator, then all simulators function as observers by default. It should be noted that the text channels are still available in this mode.
144