Comparison of Computational Time

In this section, we look at the improvement in computational time from using the ap- proximate inference methods compared with the MRF model, which uses the junction tree algorithm for exact inference. We show that as the size and connectivity of the network increases, the computational time required for the BL and SMC methods are far smaller than the MRF model. The computational cost of exact inference in the MRF model grows exponentially with the tree with of the graph, whilst the BL and SMC methods grow poly-

CHAPTER 5. SEQUENTIAL MONTE CARLO WITH BAYES LINEAR 119

(a) SMC Optimal Resampling (b) SMC Multinomial Resampling

(c) SMC Max-Weight Resampling (d) BL IS,M = 1000

(e) MRF IS,M = 1000 (f) BL IS,M = 4000

(g) MRF IS,M = 4000 (h) Bayes Linear

Figure 5.5.5: 25 node network with observations sampled using the greedy heuristic in the MRF model. The distribution of differences between inference in the Markov random field using prior conditional A and the approximate inference methods over 50 sets of random observations, given the relevance of the nodes. The SMC models give the best approximation to the MRF model for this network.

nomially with the number of nodes in the network.

The improvement in computational time will be dependent on the type of network the algorithms are run on. The MRF model using the junction tree algorithm will be faster for sparse networks, where the nodes form in small clusters. These networks will have a small tree width. Whilst for networks with no defined clusters, or dense networks, the networks will have a large tree width. Hence, to compare the computational efficiency, we simulate small networks using two different methods, where the number of nodes and connectivity

CHAPTER 5. SEQUENTIAL MONTE CARLO WITH BAYES LINEAR 120

(a) 15 nodes, p = 0.1 (b) 15 nodes, p = 0.25 (c) 15 nodes, p = 0.5

Figure 5.5.6: Networks with 15 nodes where each node is connected with probability p to each other node in the network.

can be controlled.

For each combination of parameters used in the methods, 10 networks are simulated. The true node realisations for each network are simulated using the method in Section 4.6.1, with

ρ= 0.5. The prior clique factor in Table 5.5.1a with [λ1, λ2] = [0.5,0.5] is used to define the prior MRF model for theZs. The prior mean is set to 0.5 and the prior covariance used in the BL and SMC methods is defined using the method in Section 4.6.2 withρ = 0.8. The prior conditional beta distribution is given in Table 5.5.1b.

On each of the networks, the greedy decision algorithm is run for 100 iterations. We compare the average computational time taken for a single iteration of the decision algorithm (over the 100 iterations on the 10 networks).

The first method we use to simulate the networks has two parameters: the number of nodes,n, in the network and the probabilityp that one node is connected to another node. A higher probability of being connected will result in a more dense network. Figure 5.5.6 shows example networks withn= 15 nodes and probabilities p= (0.1,0.25,0.5).

Figure 5.5.7 shows the log computational time in seconds for between 10 and 30 nodes and a probability of each node being connected set top = (0.1,0.25,0.5). With only 10 nodes, the computationally time to perform an iteration of the decision algorithm in the MRF model is less then the other two models. However, as the number of nodes increases the two approximate inference methods become more computationally efficient than the MRF model. For higher probabilities, p, the computational time for a single iteration of the

CHAPTER 5. SEQUENTIAL MONTE CARLO WITH BAYES LINEAR 121

Figure 5.5.7: The log computational time in seconds for a single iteration of the greedy decision algorithm averaged over 10 networks with 100 iterations, for networks withnnodes and probabilityp that a node is connected to another node.

MRF model increases exponentially as the maximum tree width will soon be similar to the number of nodes in the network. The computational time for the BL and SMC models does not depend on the tree width. They increase polynomially with the number of edges in the network. The computational time of the SMC model is always slightly longer than the BL model.

The junction tree algorithm used in the MRF model will be more computationally efficient when the nodes in the network group together in small clusters. For networks simulated using the first method, this property is not taken into consideration. The second method we use to simulate networks is therelaxed caveman graphfunction in NetworkX (Hagberg et al., 2008) which has 3 parameters: The number of cliques, I; the number of nodes, k, in each clique; and the probability each node is re-wired, q. The junction tree algorithm is more suited to networks when q is low. The model can split the network up into small cliques. Figure 5.5.8 shows a network with 5 cliques and with 4 nodes in each clique where the probability of rewire is 0.2 and 0.6.

CHAPTER 5. SEQUENTIAL MONTE CARLO WITH BAYES LINEAR 122

(a) I = 5, k = 4, q = 0.2 (b) I = 5, k = 4, q = 0.6

Figure 5.5.8: Networks simulated using the relaxed caveman graphfunction.

Figure 5.5.9 shows the log computational time in seconds to run a single iteration of the greedy algorithm for networks simulated using therelaxed caveman graphfunction. Figure 5.5.9b shows the results withI = 4 nodes in each clique and Figure 5.5.9b shows the results with I = 8 nodes in each clique. With 4 nodes in each clique, the SMC model has the highest average computational time for nearly all values of k, apart from k = 10. In the case where k = 10 and q = 0.3 the computational time of the MRF model has increased above the other two models.

The BL and SMC methods do not depend on the probability that each edge is re-wired. However, the probability of re-wire does effect the computational time of the MRF model. With a small number of cliques, the MRF model is the most computationally efficient but as the number of nodes increases the computational cost eventually rises above the BL model. For higher re-wire probabilities, this increase in computational cost increases at a faster rate as the tree width of the graph increases. For 8 nodes in each clique, the computational time for the BL and SMC methods remains short whilst the computational time for the MRF model quickly increases as the number of cliques increases.