Limitations

The main limitation of our user model design method is that the evaluations are based on self-reported user data. It is difficult to assess whether responses are reliable or if they are affected by other factors that haven’t been considered.

An argument emerges that it is not sufficient to assume that users are capable of understanding their own performance and potential. Therefore, responses tend to be subjective to the particular person and the situation (s)he provides them.

However, we addressed this issue by (a) collecting data from many users (52) so that the results are statistically significant and (b) carrying out user profil-ing tests to first check that our user models are varied and to identify any patterns or irregularities between their responses (Fan et al.,2006).

Although, there is still the controversy that the Likert-scale self-reports that we employed might be fundamentally an invalid method for assessing the users. Thus increasing the amount of subjects will not compensate for any biases. A solution would be to generate the new paths through the decisions and monitor the users when driving their new personalised tracks. This will provide us with the data to validate the implied characteristics of our Framework through the direct comparison between the user’s performance and engagement in different trials.

4.6 conclusion 119

4.6 conclusion

In this chapter we described the implementation and validation an adaptive user modelling Framework for transforming user’s low level primitives com-posed of game-related user actions, game outputs and unobtrusive physiolo-gical data such as head pose and eye tracking into theoretical frameworks of learning and development (e.g. concept of flow andZPD). Our intention is to provide a flexible algorithm that identifies the on-line weaknesses and performance of a race driver so that we can alter the track’s path to fit the individual’s level of skill. The functionality of the framework so far is to provide segment alteration instructions (i.e. same, easier, challenging) and also point out the human factors, that lead to the particular decisions.

Real user experiments using a simulator setup allowed us to calculate the correlations of our extracted features (Low Level Primitives) and estimate the engagement of the users to the task, through their comparisons to an expert – a trained and self-engaged – user. In addition, simple and logical rules regarding the racing task were embraced in order to perform theTT’s transformation process of low level traces to the theoretical frameworks. The fitness of the rules and in general the modelling process was verified by associating user responses to the experiment with the offline instructions generated. User profiling helped us understand the variation in the user types whereas feedback helped us to improve our future experiments. An interesting result is that our user responses were not found to be statistically different between genders as indicated inBacklund et al.(2006) as well.

Over the next Chapter we will show the results for verifying our model through experiments where the track is being altered through the decisions of the model in real-time and the user is going to be able drive the new track straight away. User’s attention, skill and challenges will be monitored with the aim to be kept above a threshold through the whole process. This approach will also validate the Framework without the need of user responses.

As far as we are aware, this kind of experiment hasn’t be conducted before.

However, the forthcoming main challenge is to elicit a new path that fits the user model. The path construction algorithm has to include both the instruction generated and the specific human factors that lead to the decision in order to create new segments that will aid in both engagement and training of the user.

An important future aim will be the adaptation of that model structure to another game or task through minimal changes. This will prove the

transfer-ability of the model with as few modifications possible and show its generic capabilities to capture the engagement and enhance the training of the user.

P E R S O N A L I S E D T R A C K D E S I G N I N C A R R A C I N G G A M E S

5

In the previous Chapter, we designed, implemented and validated through user feedback a user adaptive model for car racing games. We proposed a Framework where a combination of raw data from the game and sensors provided (e.g. eye tracking and head pose) are used to extract relevant features through machine learning techniques to implement a user model that is able to explain the current user’s performance during gameplay. In this Chapter, 5, we further enhance our Framework with the aim to:

1. monitor the user’s state and properties in real-time 2. update the user model

3. provide decision adjustments for the alteration of the racing track according to the user

4. generate new tracks that suit the user profile.

This section is divided into two parts: The first describes the overall data-flow of the track design algorithm. There is a an overview of the user model de-scribed in Chapter4along with a detailed explanation of how the new tracks are expressed and designed in the game. The first part ends with preliminary results using the data collected in Chapter4 to show the applicability and diversity of the generated tracks according to the time performance of the user. The second part concentrates on the evaluation of the framework by extensively testing the evolved tracks with human subjects. The results presen-ted in the first part of this chapter are based on our previously published work (Georgiou and Demiris,2016).

5.1 system and data flow for track design