Search Control Techniques

A number of additional techniques are used by modern Chess programs to use the time available to them effectively and to search in a “smarter” manner. In this section, we review three of these techniques — iterative deepening search, forward pruning and selective search extensions.

Iterative Deepening

One key drawback of fixed depth alpha-beta search (and its variants), as pre- sented in Algorithm 2.2, is the unpredictability of its runtime. This is particu- larly problematic in games like Chess where strict time controls are employed, since the search procedure may not complete in the allotted time. Alpha- beta search is thus combined with iterative deepening, to make the algorithm “any-time” [8]. Starting with a 1-ply search, the current position is repeatedly searched with deeper depth bounds as each previous search terminates, un- til time expires, i.e., from the current position, the program carries out a 1-ply

search, a 2-ply search, a 3-ply search and so on. Due to the exponential growth in the size of the trees being searched on each step, the overall run-time of iterative deepening depth-first search is dominated by that of the deepest search, and the runtime overhead compared to a single, deep search is not significant. In fact, using the outcome of shallow searches to order moves for deeper searches of- ten means that the iterative deepening search terminates faster (or can search deeper).

Forward Pruning

The pruning methods we have encountered so far have all been sound, in that they provide the same result as a full-width Minimax search. In his seminal paper from 1950, Shannon termed this brute-force approach ’Type A’ [94]. In the same paper, he described another approach one could take to programming a Chess playing computer — the ’Type B’ programs. Rather than explore every possible move at each level of the search tree, these programs were to be more selective in the lines of play that they would search deeper. Indeed, most early Chess playing programs were of Type B. However, as the speed of computing hardware increased, Type A programs such as CHESS 4.5 [8] and TECH [48] came to dominate the field. The Type B programs were handicapped by the fact that the problem of heuristically determining the utility of exploring a certain line deeper was unreliable and slow [1, 17, 64, 108].

However, there are a few forward pruning techniques that are employed by many modern Chess engines. Null move pruning [12, 34] is applied to nodes one ply up from the leaves of the tree (known as frontier nodes). It determines whether the opponent’s position is strengthened by allowing the current player

to “pass” his turn; if not, the current node is not expanded further. This ap- proach assumes that doing nothing on one’s turn always hurts the player’s po- sition, although this is known to generally not be true. Many games, including Chess, have so-called zugzwang positions where the best move is to pass one’s turn, if the rules of the game allow it. Futility pruning [88] is also applied to frontier nodes: it works by bounding the expected change in the evaluation of the current position, and determines if this is likely to fall outside the current α and β bounds. If not, then searching further from this position is deemed futile and not pursued. Finally, Michael Buro’s ProbCut technique uses offline statis- tical models built from a large database of positions to guide in-game pruning decisions. While it has produced mixed results in Chess, it has been quite effec- tive in LOGISTELLO, one of the world’s strongest Othello programs [22].

Quiescence Search and Selective Extensions

Game playing programs, particularly for Chess, are susceptible to the Horizon Effect [14]. This occurs when a program mistakenly believes that an inevitable catastrophe can be avoided by making a series of ineffective (and potentially, more damaging) delaying moves. For example, consider a position where a player will lose his queen in eight plys, or if he chooses an immediate rook sac- rifice, will lose the queen in twelve plys. A program that only looks ahead eight plys in this situation will decide to sacrifice the rook, thinking that the queen has been saved; however, the loss of the queen has simply been postponed to a point beyond the search “horizon”, and the program is now in an even weaker position having unnecessarily lost a rook as well. This problem was identified very early on by Turing [109] and Shannon [94] who recommended waiting for

positions to become “quiet” before applying static positional evaluation meth- ods. Every good Chess engine today employs a quiescence search routine, that ensures that the current position is stable before evaluating it; this entails check- ing for potential captures, promotions and kings in check.

Another approach that is used to search unstable and interesting lines of play deeper is the use of selective extensions. One common approach is to keep count of the number of “interesting” moves that were made as we descend a line of play. If this number exceeds a pre-determined threshold at the leaf node, then this line is searched for another ply or more. This technique is referred to as fractional ply extension [58]. Another approach, that was used in DEEP BLUE, is termed the singular extension [6]. This technique extends the search depth of moves that are deemed singular, i.e. moves that produce positions whose evaluations are better (or worse) than that of all their siblings by a significant margin. Such moves are usually indicative of interesting positions worthy of deeper analysis.