Pac-Man Weighted (PW) Heuristics

The Pac-Man algorithm introduced in this section utilizes a summation of weighted heuristics to determine the next move of Pac-Man. Each heuristic calculates infor- mation about the world that the agent is capable of viewing and produces a score for that observation. Based on a sum of these heuristics a resulting score will be produced. A cumulative score will be produced from the heuristic scores for each of the four possible directions the agent could move, the position with the highest cu- mulative score is selected as the next move. Each heuristic has an associated weight this number represents its priority or inuence for a particular heuristic. Each heuris- tic has its own weight. In the case of a tie, a random direction is chosen from the highest tied results. Throughout the course of this algorithm we will discuss the use of distance. In our testbed distance was measured utilizing the Manhattan distance function [38]. The Manhattan distance indicates the minimal number of squares used to traverse between two positions. The Manhattan distance improves the estimates of movement distance over Euclidean distance as Pac-Man and the ghosts can not move diagonally. Pac-Man's heuristics are can be organized into to three categories; edible goals, avoiding ghosts and global positioning.

Pac-Man's edible goals are based on the distance to the following set of game objects: tokens, power-pellets, fruits and any edible ghosts. Each distance function will be weighted to specify the goal's importance to the game strategy. As tokens and power-pellets must be collected to progress past a level they represent an essential role in the game. As such we expect tokens and power-pellets to be important whether prioritized or not. Our interest in prioritizing these objects was to observe possible

scenarios where a player would attempt to completely clear an area before moving to the next area or would prioritize power-pellets and move quickly between them leaving a large number of tokens behind. Pac-Man's accomplishments of eating fruits or ghosts are rewarded in the form of extra points. In addition, Pac-Man eating ghosts provides a strategic advantage as well, as the ghosts are temporarily removed from the game board. The bonus fruit can be placed anywhere on the game board causing the player additional diculty to pursue the extra points. Weighting heuristics for each of these four game elements potentially represents dierent player strategies. When the weight for tokens and power-pellets is high the player is focused on completing the level. A high fruit weight represents a player who is seeking a higher level of challenge from the game. If a weight for power-pellets is high, yet the token and edible ghost weights are low, this scenario may highlight a player who is struggling with the game and attempting to only use the predator mode to collect tokens.

Pac-Man must avoid ghosts while playing the game, thus a distance and direction function is used for each ghost. The weights for these functions represent the player's comfort level for approaching ghosts. The highest level weights represent a player unwilling to head in the direction of ghosts unless absolutely necessary. A low level potentially represents a player less concerned with the close proximity of the ghosts positioning.

The nal section introduces heuristic for Pac-Man's global positioning or at least positioning based on the visible board. There are two global positioning heuristics, one which keeps Pac-Man away from the centroid of the ghosts, and another which moves Pac-Man towards the centroid of the remaining items. For each of these heuristics, centroids are calculated based on visibility.

The weighted heuristics algorithm was inspired from the work of Yannakakis and Hallam [53] and later modied due to research results of Szita and Lorincz [50]. Yannakakis and Hallam utilized a greedy algorithm for Pac-Man's strategy. In their version of Pac-Man, each square was given a value, such as 0 for squares occupied by tokens, 10 for empty squares and 100 for squares occupied by ghosts. Pac-Man would choose squares which minimized the value of his next move. After initial trials

to produce a base level of performance, they included two additional rules. The rst improved global token consumption by moving towards the closest token if all neighboring squares were empty. As well, they included an additional ghost avoidance rule to help avoid traveling in the direction of visible ghosts. There were a couple of issues not addressed within the Yannakakis and Hallam's version of Pac-Man which needed to be addressed in our version of the game. Their version of Pac-Man did not include power-pellets or the bonus fruits as they deemed those items to be less important to the level of player interest. Thus our version of the game includes additional heuristics to account for dierent values for eating tokens, power-pellets and fruits. Also our version allows for Pac-Man to become a predator which means that moving towards ghosts can be benecial to both strategy and score.

Szita and Lorincz [50] created a list of action modules for the creation of rules within their version of Pac-Man. Action modules are actions which will become prioritized as a result of observations made by the player during play. For our version we are not attempting to learn while simulating, but simply need the actions and an approximation function for evaluating the eectiveness of performing each action at a specic point in the game. From their results we utilized the actions from the two most successful learned policies. The actions included are: moving towards and away from power-pellets depending on whether Pac-Man is a predator, moving towards edible ghosts, moving towards the center of items and moving away from the center of predator ghosts.

A brief summary for the factors selected for the Pac-Man weighted (PW) algorithm are available in Tables 4.5 and 4.6.