Dynamic Scripting

Scripts are used in games to describe the actions, behaviors and attributes of an object. The advantage of using scripts is that they provide a clear explanation of object behavior, and are highly adaptable. However, scripts can become quite large, resulting in decreased speed of execution, and increased diculty debugging [39]. Overusing scripting can result in a less compelling game world, if there is not enough variety of events or actions. A common usage of scripts occurs in role-playing games (RPG's) where a neutral non-playable character continually performs the same actions and repeats the same conversation. Dynamic scripting allows changes to opponents and other non-playable characters (NPC) behaviors during runtime.

Dynamic scripting is based on reinforcement learning, where behaviors that are unsuccessful are punished and successful behaviors are rewarded. Dynamic scripting can be applied to games with the following three requirements: the game AI can be scripted, the domain knowledge on the characteristics of a successful script can be recorded and an evaluation function can be created to assess the success of scripts [42, 43]. The domain knowledge database from which rules are designed is usually

hand crafted by game designers. Rules dictating the behavior of a character are chosen online based on the probability of a weight associated with that rule. Recent research has attempted to automatically generate scripting rules for the database, but for the most part rules are manually edited [34, 35]. A given rule's weight is altered based on the success and failure of that rule in previous situations. If the rule succeeds the probability of selecting that rule is increased, and if the rule does not achieve a positive result, its value is lowered. This process ensures that the most challenging and successful rules have a higher probability of being chosen.

Dynamic scripting forces the player to develop dierent sets of strategies, because it specically chooses rules that the player has struggled with. The process of increas- ing the probability of selecting successful rules is not a full auto-dynamic diculty system, as the challenge continually increases. A method of allowing the level of chal- lenge to decrease using dynamic scripting is known as diculty scaling[44]. Diculty scaling was integrated in the dynamic scripting framework by restricting rules that have continually proved to be successful against a player. Including diculty scaling results in database rules being pruned because of two reasons: they rarely achieve success or they consistently achieve success. Thus the database allows a subset of rules to be selected, these rules are within the player's capability, and should provide a balanced result.

Game diculty can be scaled based on three methods: high tness penalizing, weight clipping and top culling. Diculty scaling proved to be eective at balancing the game against a variety of static strategy opponents [44]. High tness penalizing awards the highest score to the technique that provided the closest competition. Thus, strategies with the lowest variance between the players are evolved as the ideal strategy. Weight clipping provides a window of strategies that are available; the size of the window is based on a weight. If the weight is low, indicating low performance, the window is made larger providing a larger variety of strategies to be chosen. Inversely, if the weight is high, the window provides a smaller number of strategies to be chosen. Top-culling is a window based approach where extremely successful techniques are culled. Once a player defeats medium level strategies, higher level strategies become

reactivated. Top-culling resulted in the lowest variance in the number of wins between the three strategies. It was also the most successful at balancing the number of wins against weaker opponents.

An outcome of dynamic scripting is that it adapts to continuous exploitation of a particular strategy automatically. Without dynamic scripting, when a player uses a certain strategy that always works, they will use that strategy to cheat the system. The ability to exploit the same strategy continuously aects the enjoyment of player types in dierent ways. Conquerors and managers may experience this deviant gamer behavior as an immediate progress method, but overall it decreases the enjoyment of the experience, as it eliminates the need to discover new strategies and solve problems, thus reducing the sense of accomplishment for beating a game. On the other hand, wanderers and participants are often more interested in the development of characters and the story. These players will gain more from progressing the story than from the accomplishment of defeating the game in a meaningful way.

One of the main criticisms of dynamic scripting is that it requires a reduced state and action space to meet the eciency requirement from the RAGI requirements [34]. This limits the use of the dynamic scripting from certain game genres with larger state spaces. A highly benecial addition to dynamic scripting has been automatically gen- erated game tactics. Typically, scripts are manually edited by programmers and game developers; this is expensive in the number of hours required for development. Since dynamic scripting weights dierent scripts based on their performance, it is limited by the number of initial strategies implemented in the rule base. This means that if the player's abilities are outside the range of initial script strategies, dynamic scripting will not be able to adapt to the player. Previous research by Molineaux and Ponsen [3] built a case-based reasoning system for evaluating the eectiveness of techniques for a given state and opponent. Ponsen et al. [34], investigated automatically gener- ating game tactics through the use of evolutionary learning, where each game state contained a corresponding available action that was related to research, economy, combat or building. The resulting evolved tactics were compared to a set of scripts not in the training set, in a new game world to test whether evolved tactics could

outperform a variety of strategies. Results showed that dynamic scripting was ca- pable of consistently defeating all but the strongest scripts, which provided an even level of challenge for dynamic scripting.

Chapter 4

Experimental Testbed Design

This chapter provides extensive insight into our dissection of the problem of auto- dynamic diculty for our research purposes. Section 4.1 provides an overview of decisions made during the design process to reduce the problem to a manageable size. Section 4.2 provides an introduction to Pac-Man, the testbed game selected for our research. The remaining sections in this chapter will provide a detailed view of the design of the game Pac-Man as utilized in the experimentation phase. Section 4.3 introduces the initial phase of the game design in which we developed and priori- tized a detailed list of game factors that our experimentation stage would investigate. Section 4.4 introduces strategies used to represent player behavior, followed by Sec- tion 4.5 which introduces algorithms used to dictate opponent behavior. Section 4.6 introduces and discusses performance measures which will be tracked and used to understand the performance as well as predicting the form of heuristics. Finally, Section 4.7 introduces the adaptive system and its method of predicting performance and performing adaptation. The selection of response variables to be controlled via the adaptive system will be left until the analysis phase presented in Chapter 5 as the selection process is dependent on the performance results.

4.1

Adaptive Gaming Architecture Design Overview

The design phase of our research is conducted from the perspective of GameFlow and the USE model with the long term goal of studying auto-dynamic diculty. The USE model, reviewed in Section 3.1, provides a high-level view of user interactions with the system. The USE model was designed specically for explaining relationships between users' experience and video games. A key feature of the USE model is the

separation of usability and playability while being capable of describing multiple types of usage experiences.

Within the USE model we wish to address the issue of an adaptable system; thus, we narrowed the design focus to the adaptive game system (AGS) framework. As discussed in Section 3.2, the AGS model is composed of four main components: mon- itoring a player's performance, adapting in response to a player's needs, monitoring the eectiveness of adaptation, and dynamic player modeling. Monitoring the player's performance can be task specic, and unique to each game. Often, heuristics are de- vised to approximate the level of diculty the player is experiencing in performing a task or playing the game. As explained in Section 3.3, although some heuristics may appear as obvious factors to the level of diculty, they can be misleading [7]. The fourth component of the AGS model deals with the dynamic remodeling of the player types, which ideally requires being able to identify a player's player type solely by ob- serving game play, but could be aided by utilizing other intrusive research techniques such as pre-game questionnaires. Our research will focus our experimental design on two components of the AGS framework: adaptation in response to a player's needs and monitoring the eectiveness of adaptation.

The main task of performing auto-dynamic diculty is maintaining an adequate level of challenge for the player's level of skill. Thus our focus will be on adaptation in relation to the player's level of challenge, as well as the perceived level of challenge. Our research will focus on the two main types of playability issues: challenge and curiosity. Focusing on the level of challenge for adaptation purposely lters out some of the player's needs, such as when to provide feedback and what type of feedback. Our focus on challenge investigates the player's current scenario and provides within- subject information relating all of the game possible factor settings.

Our design will focus on identifying important factors for adaptation in the game environment which could potentially be used in response to a player's known needs. Our design focuses on being applicable to both the research and commercial com- munities. In the research community, NPCs utilize machine learning techniques that require a large number of training iterations in which the game usually trains against

itself to produce adequate behaviors. However few commercial games are distributed with the ability to adapt outside predened tested intervals. One major issue in cur- rent adaptable game system research is the ability to quickly adapt from a novice skill level to an expert level. Our approach hopes to provide additional information to help understand which factors have the greatest inuence on the diculty of the game, such that the NPC's learning process could more eectively bridge larger gaps in skill level. The same information could be utilized in the commercial community by game experts specically to tune factors for level design issues.

We utilized a closed experiment where adaptation occurs oine between simula- tions. An oine setting creates a controlled environment, allowing inspection of each game factor to occur individually from the beginning of a new game. Throughout game play, all major game event information will be logged for evaluation. Logged information will be utilized during a post-game analysis phase in which we will de- termine each factor's statistical signicance to the level of diculty.