4.2.1
Iteration Budget Results
The mean average results of the state S1experimentation are displayed in figures 4.3 and 4.4.
The values for variance of each of these averages are very small (< e−13), due to the high
number of repeats.
There is a negligible difference between using Tree and Tree with VL in both MCTS
and ISMCTS. As there is little overhead to adding or removing the virtual loss, then the
main difference in speed would be seen when a virtual loss fails to cause a different node to
be selected by the selection policy, as this means that a thread would contest a mutex. The 1http://en.cppreference.com/w/cpp/thread/mutex
fact that the results for both are nearly identical suggests that the virtual loss makes little
difference to selection, and this is confirmed by additional testing which shows that virtual
loss is causing a selection difference less than 1% of the time. So either the selection choice
is very clear and a single loss is not affecting the choice, or that the choice is very unclear
as the statistics are similar in most nodes at a given level, and nodes are effectively being
chosen at random. Chaslot et al. [37] reported that Tree with Virtual Loss performs as well
as Root Parallelization on smaller boards in the game of Go, but this does not seem to be the
case with Lords of War.
Leaf parallelization is clearly a far slower technique than any other used here. Using
more than 3 agents does not result in a speed increase. From the results in figures 4.3 - 4.6,
we can see that the addition of agents numbered above 3 has almost no effect on the results:
the simulations assigned to earlier agents are already complete by the time a simulation
would be assigned to an agent numbered 3 or higher.
Tree Parallelization shows itself to be a competitive technique in terms of speed (fig-
ures 4.3 and 4.4), but still a lot slower than Root Parallelization in both MCTS and ISMCTS.
If we calculate the difference in speed between Tree and Root in MCTS, then the difference
in speed between Tree and Root in ISMCTS, it can be seen that the difference is compar-
atively lessened in ISMCTS, but that the speed decrease caused by ISMCTS is still more
significant.
As discussed earlier, we can see the effects of using mutexes to lock nodes by comparing
the difference in performance between root and tree parallelization when using 1 agent,
however this only accounts for the actual cost of the locking procedure, not the expense
caused by causing any threads to wait. The average of this difference is very small, the
best estimate being less than 16ms (due to the resolution of the timer used). This indicates
that the time spent locking mutexes is very low, and almost all of the expense comes from
0 200 400 600 800 1000 1200 1400 1600 1800 2000 1 2 3 4 5 6 7 8 Decisi on Comple ti on Ti me (m s)
Number of Agents (Threads)
LEAF (UCT) TREE (UCT) TREE VL (UCT) ROOT (UCT)
Figure 4.3: Results of applying four different parallelization techniques to UCT operating upon state S1. Results are expressed in milliseconds used to complete a decision.
ISMCTS runs of root and tree parallelization.
In order to see the relative effects of different parallelization techniques on UCT and
ISMCTS, we can compare the relative efficiency of individual agents within each technique
(see figures 4.5 and 4.6.) Efficiency is calculated as t1
n·tn, where tnis the decision time for n
agents. In particular, the efficiency for n = 1 is t1
1·t1 = 1.00. In an ideal scheme with 100%
efficiency, using n agents would result in an n-fold increase in speed: adding the second
agent would cause overall speed to double resulting in a decision time of t2 = t21, and so on.
The results show that root parallelization spreads the load between agents most effectively,
0 200 400 600 800 1000 1200 1400 1600 1800 2000 1 2 3 4 5 6 7 8 Decisi on Comple ti on Ti me (m s)
Number of Agents (Threads)
LEAF (ISMCTS) TREE (ISMCTS) TREE VL (ISMCTS) ROOT (ISMCTS)
Figure 4.4: Results of applying four different parallelization techniques to ISMCTS operat- ing upon state S1. Results are expressed in milliseconds used to complete a decision.
4.2.2
Win Percentage Results
The results of the state S1 win percentage experimentation are displayed in figures 4.7
and 4.8. The general curve of the results shows that each iteration added is less effective
than the previous, and thus has a smaller effect on play strength for our fixed 5000 iterations.
As in the previous experiments, we can see that there is little difference in behaviour
between Tree with and without Virtual Loss. As mentioned previously, we can attribute this
to differences in the game of study, and also that a single temporary loss is unlikely to affect
overall decision making, as the difference is intended to have a minimal effect on algorithm
operation.
We can see that tree parallelization performs substantially better in these experiments,
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 2 3 4 5 6 7 8 Indi vid ua l Ag en t Ef fi ciency
Number of Agents (Threads)
LEAF (UCT) TREE (UCT) TREE VL (UCT) ROOT (UCT)
Figure 4.5: Results of applying four different parallelization techniques to UCT operating upon state S1. Results are expressed in individual agent efficiency, where 1.0 represents optimal efficiency.
unlike in Root parallelization where processor time may be wasted re-exploring decision
space already explored by other agents.
We can see that overall ISMCTS behaves more poorly than UCT in terms of efficiency,
however this is to be expected, as the UCT agents have access to perfect information of the
game state, and the ISMCTS agents include enhancements to cater for the lack of such in-
formation, so this direct comparison does not speak of their relative effective play strengths.
Leaf parallelization performs approximately equally to standard UCT, flattening out al-
most immediately after adding additional agents. This indicates that the simulation process
is not significantly expensive for this game, and thus parallelizing this process alone does
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 2 3 4 5 6 7 8 Indi vid ua l Ag en t Ef fi ciency
Number of Agents (Threads)
LEAF (ISMCTS) TREE (ISMCTS) TREE VL (ISMCTS) ROOT (ISMCTS)
Figure 4.6: Results of applying four different parallelization techniques to ISMCTS operat- ing upon state S1. Results are expressed in individual agent efficiency.
tion types (see figures 4.7 and 4.8).