Third play test session

The third and final play test session was classified as a beta version of the prototype as the game was viewed to be feature complete. The beta version of the ThreatBot prototype is therefore very similar to the final ThreatBot prototype which was discussed at length earlier in this chapter. Significantly, it was the play test to include not just the Default bot AI but the then recently finalized and enabled ThreatBot AI. The third play test was primarily used to get written and verbal feedback on the following list of game elements:

 New damage boost pickup (audio, effects and balancing)

 Bot difficulty scaling - bot's difficulty adjusts depending on player performance

 ThreatBot AI

 Proximity Spawning System

 Degenerating HP/Armour

A total of eight play testers generally reported positive feedback from the new components of the 3rd play test. Additionally, play testers reported that their game

184

against the ThreatBot AI was the game they preferred, even if they were not sure which game had which opponent type. Only a small number of suggestions were made regarding weapon and pickup balancing such as reducing the bonus of the damage boost pickup as well as increasing the area of effect of rocket explosions. These changes were applied for the final version of the game prototype.

In addition to testing new content, the third play test was also concerned with evaluating the difference in difficulty between the two bot types, especially when difficulty scaling was enabled. The average difficulty of both bot AI types was recorded during play tester matches to see whether there was a large difference in difficulty between the AI types both overall and per play tester. As explained in section 8.2.2 of this dissertation, bots start at a difficulty of five and increase or decrease in expertise depending on the performance of the player. If the Default or ThreatBot AI was found to be significantly harder or easier than additional tuning would be necessary to ensure equilibrium could be attained.

While there is a large difference in performance between play testers, for most of them the difference in average and mode difficulty experienced was roughly the same. Overall, the average difficulty experienced between the eight play testers across all 16 games was a difficulty of 4.2 for the Default bot and 4.5 for the ThreatBot. While this suggests that the Default bot is slightly more difficult, the average mode of difficulty experienced across the 16 games was five for the Default bot and four for the ThreatBot. Additionally, two of the eight play testers did slightly better against the Default bot, with six of the eight doing marginally better against the ThreatBot. However, the highest average difficulty difference between the two AI types experienced by a play-tester was only three difficulty levels, undertaken by play tester five. While this play tester experienced the Default bot being more difficult than the ThreatBot, they described their ThreatBot game as "in their favour". As it was observed that play tester five was the most skilled FPS player amongst the play testers, this episode of dominance was not considered unusual.

Overall, only two play testers managed to beat either the Default or ThreatBot variants during their games, with play tester eight beating both types and the play tester five tying with the Default Bot and beating the ThreatBot. The play testers did not believe the bot AI to be too difficult as most felt they were trading consistently

185

with both AI types. As bot difficulty scaling was rationalised as creating a maximum number of encounters with the bot, bot difficulty scaling was considered a success. Additionally, while play testers were not told to notice any difference between the behaviours of the AI types, several play testers did comment on the difference in decisions being made. In particular, play tester two reflected verbally that they "could not get a handle on when the second bot would run away, which made the bot more enjoyable to play against". With the results from the bot difficulty normalization viewed to be successful as well as the vast number of bug fixes completed, the final user study of the research could commence.

186

9 ThreatBot User Study

The final user study of this PhD research project was conducted as a double- blind user study using two separate builds of the ThreatBot Game Prototype. These builds were slightly modified versions of the master build of the ThreatBot Game Prototype discussed in detail in Chapter 8 and were used to accommodate for different participant usage during the ThreatBot User Study. The objective of this study relates to the third and final research question derived from Aim 3. Aim 3 and the third and final research question are again the following:

Aim 3: To develop a prototype system of bot AI that builds on the identified model to measure levels of player enjoyment.

3. To what extent does a developed player-like AI model applied within a working FPS game improve player enjoyment?

The developed ThreatBot Game Prototype, detailed in Chapter 8, was used in the ThreatBot User Study to measure the differences in enjoyment between a Default and ThreatBot AI type variants. As discussed in detail in section 8.2 of this document, the main differences between the Default and ThreatBot AI types involve how decision making is performed with respect to a calculated level of threat experienced. Participants of the final user study are required to compete against the Default and ThreatBot AI types, detailing their preferences and describing their enjoyment through the use of a questionnaire. The qualitative and quantitative data from this questionnaire provides insight into the third research question, as well as determine whether the primary hypothesis underpinning this research is true or not. This research hypothesis is again the following:

First Person Shooter Bot Artificial Intelligence designed to reduce predictable and unrealistic behaviours will be more enjoyable for players than traditional FPS Bot AI.

Given the context of the research – bot AI within a FPS game – the focus, with respect to enjoyment, is on challenge and a player‟s feeling of competence in the face of challenges. As highlighted in Section 5.5, feeling threatened in the face of a

187

challenge may be related to the level of skill a person is able to apply in addressing that challenge. Self Determination Theory (Ryan & Deci, 2000; Ryan et al., 2006) identifies feelings of competence as a being core to motivation, and Flow theory (Csikszentmihalyi, 1975) identifies the relationship between skill and challenge as being pivotal in producing deep and energised focus on an activity. Further details on the measurement of enjoyment are included in Section 9.2.