world to the simulated data to improve the model-based dynamics and also improve the policies.
2.5 conclusion
In this Chapter we reviewed physiological concepts of learning and develop-ment as well as engagedevelop-ment and how these are connected to specific tasks.
We have also described several user modelling approaches and frameworks in games and driving in particular.
Since our training and engagement of the user is based on personalising the content of the environment, we described the relevant frameworks in gener-ated adaptive contents and their utilisation in games. Finally, we presented a review of behavioural models concentrating on driver’s prediction and stated the capabilities of the algorithm we used for our own prediction model.
In the rest of the thesis we will show how our broad area of relevant background was employed to build the user models we are using to train, engage and predict the user.
S Y S T E M D E S I G N A N D F R A M E W O R K F O R U S E R
3
M O D E L L I N G
Our aim was to provide a system with a realistic and immersive virtual car-racing environment that outputs real-time data from our user’s actions. This would make us able to analyse and develop models for human behaviour under high performance driving. The composition of our system is of high importance since our goal was to contribute towards a general framework for user modelling, training and prediction.
As shown in Figure 3.1the framework consists of three basic systems:
• The Simulator which consists of the car simulator’s related hardware and software as well as the eye tracking unit.
• The Track Design system which conducts the manipulation and alteration of the simulator’s tracks.
• The Core which handles the storage, analysis, and implementation of the user model, as well as the real-time display of important events. It is also used as a clock synchronisation server through Network Time Protocol (NTP) for the other systems.
Over the following Chapter we will go through each system and clarify the hardware utilised, the software and algorithms employed as well as the major data flow of our system architecture.
3.1 the simulator
User experience and collection of truthful data is always of primary import-ance in research. Thus the Simulator is equipped with demanding hardware specifications to provide the best graphic performance and avoid the genera-tion of any game lag. As shown in Fig.3.2a, a custom set-up of the Vision Racer VR3 seat is used, with a Logitech G27 Force Feedback Steering Wheel, three pedals, a gear shifter and a combination of three monitors to enhance the user’s experience and thorough immersion in the game. A Tobii EyeX eye tracker is installed as well as 2 RGB-D cameras (Kinect). The two cameras are positioned in such a way that one of them captures the player’s face
57
Figure 3.1: The Framework’s systems hardware and software properties. The systems are interconnected with each other using a high speed 1GbErouter to facilitate real-time data transfer.
and the other captures both the player’s actions and the output from the monitors. Real-time performance metrics, 3D-car position and user actions are displayed on the wide screen monitor. More details on the component’s specifications can be found in AppendixJ.
The racing car simulator software employed to carry out our experiments is called rFactor1, recommended for its excellent graphics and realistic vehicle dynamics by many users and racing teams. It is accompanied with an Application Programming Interface (API) as a plugin that provides useful data about the interesting events, user inputs and car outputs at a high frequency (100Hz). It also allows off-line customisation of the game contents such as the cars and the tracks.
3.1.1 Data Flow
Figure 3.3 shows the data flow of the Simulator system while the user is driving. Both the simulator and the eye tracker’sAPIshave been modified to save their sequential data asynchronously in a JavaScript Object Notation (json) format and also broadcast them in real-time through a User Datagram Protocol (UDP) port to the Framework. A list of the data available is shown in TablesM.1andM.2in AppendixM. For configuring certain properties (e.g.
1 rFactor2 Simulator [Online]:http://rfactor.net/web/rf2/(visited on 18/08/2016)
3.1 the simulator 59
(a) The Simulator
(b) Screen View
Figure 3.2:(a)Our custom car simulator setup consisting of Vision Racer seat with a force feedback steering wheel, pedals and three monitors to enhance the user’s immersion to the game. Two RGB-D cameras as well as an eye tracker were installed to capture user’s physiological signals and the environment.(b)Screenshot of the game from the simulator’s screens.
output file locations/formats, Internet Protocols (IPs), ports) of theAPIs, we are using a local XML file that is read by eachAPIwhen invoked (its default values are shown in AppendixN.1).
Figure 3.3: The Simulator’s Data Flow: Timestamped data from the interaction of the user with the simulator and eye tracker are recorded on different files for offline analysis. Also, two UDPclients broadcast the data from the two sources through different ports to the network. A third broadcaster sends messages about the state of the simulator (e.g. current state of the simulation, error/info messages, etc.) to the network.