Custom Built Server with Arm Robot (a modified haptic device)

For further evaluation, it is important on this research to have a telerobot with full access on it. Hence, I tested this SL game to communicate with a custom built telerobot made by haptic device. The idea was to utilise an X3D application as a simulation telerobot to receive the command from SL. In addition, this application is connected to a haptic device which read the command on X3D application as a force feedback. This enabled the haptic device to act as a telerobot. The X3D is a successor of Virtual Reality Modelling Language (VRML) which is commonly used in presenting 3D computer graphics.

Chapter 2

Gaming Environments for Telerobotics User Interface

65

Due to the limitations of X3D API in communicating with the TCP protocol, which was the only way to communicate with the SL server, I utilised an XML file as an alternative mediator between the SL and X3D applications. I created a PHP server and ran a PHP script which could receive the HTTP request from SL, and at the same time update the XML file. On the other side, by merging X3D and AJAX, it allowed the application to read the updated XML file, update the 3D model in X3D application and the haptic device. This X3D application rewrote the XML file with the last position of the haptic device as the feedback. Thus, the PHP script continued to gather feedback from the XML file and send it back to the SL server. I located the XML file on a PHP server machine which could be accessed by the X3D application.

A concept of file lock was applied in order to avoid logic errors when the PHP updating process ran together with the X3D application reading process (Figure 2.32).

Is file locked? Locked XML file Process reading and updating XML- file Unlocked XML file Waiting/ looping process No Yes App-1 (SL server) App-2 (X3D) HTTP Response HTTP Request Request Response File lock command received from App-1/2

File lock scheme in PHP

Figure 2.32: File lock concept applied

Based on Figure 2.32, there are two separate closed loops which were running in the system. The locking steps are described below:

1. After receiving the HTTP request from the SL server, the PHP locked the XML file and wrote the request value into the command variable. Then PHP automatically unlocked the file after the writing process was complete.

66

2.4 Telerobot Communication by Gaming Environment

Chapter 2

2. On the other side, while the PHP locked the file, the X3D application did a looping process trying to access and read the XML file. When the file was unlocked, the X3D took over and locked the file, then received the value of the command variable as the request position to move the 3D model and the attached robot. The X3D application kept the XML file locked until it wrote a feedback response into the response variable on the XML file. 3. Meanwhile, during the second step, the PHP continued to try to gain access to read the

response variable. When the file was unlocked, the PHP would automatically read the response variable and send the HTTP response back to the SL.

During the communication test, besides the delay in getting feedback information from the HTTP response, the applied file lock concept caused additional time delay in communication. However, the remote simulation server and the robot successfully moved following the request from the SL interface (with 0.3 – 0.5 seconds in a one-way communication – for requests only). Moreover, the 3D model on SL succeeded in implementing the feedback response from the telerobot. For example, when the telerobot gave a different position from the request input because of particular condition, the 3D model would update the position according to the last telerobot position.

Based on this preliminary test, SL has a number of limitations such as: (1) requires up to two seconds in total for receiving the feedback and updating the telerobot model and (2) only allows TCP/IP protocol to communicate with remote devices. However, this gaming environment also provided satisfactory features to continue the research in evaluating the effectiveness of MR interfaces and HSC for a telerobotic based on a gaming environment. To explore the evaluation in more detail, the next chapter discusses the experiment of utilising MR concept in a gaming environment.

67

Chapter 3

Evaluation of Gaming Environments with MR

Concept

This chapter discusses further exploration of the use of gaming engines as telerobotic interfaces. A concept of mixed reality (MR) environment, which combines information from streaming video and 3D computer visualisations inside the gaming environment, was evaluated as an alternative to streaming video interfaces. This chapter discusses user performance from a number of sub-experiments from two gaming engines and the feasibility of using the MR concept in telerobotic interfaces. I demonstrate that the MR concept in a gaming engine can provide an effective telerobotic interface.