Choosing variables

When the root of a partial computation tree is computed, the system must decide which variable or variables to condition on. Since we want to avoid

conditioning if possible, this implies that we should condition on as few variables as possible, while remaining within the memory budget. As we do not know what a partial computation tree will look like until query time, we must decide this process at runtime.

In this text we use an heuristic approach of recursively choosing a variable that occurs most frequently in the largest (virtual) cliques that would be formed if conditioning were not used, until the memory budget is no longer exceeded.

This approach has the advantage that it is very cheap to compute, which is critical as we are determining the conditioning variables at runtime, and they may change on a query by query basis.

8.4.5. Example

Consider the partial computation tree in figure (8.3), taken from figure (8.2) by bridging the gap between the nodes labelledP

Aand

P

B,C. We will as-

sume our memory budget does not allow us to store the intermediate distri- bution P (B, C, D, E) that would result from P (D|B, C)P (E|B, C)P (B, C), but does allow us to store the intermediate distribution P (b, C, D, E), i.e. conditioning on (instantiating) B.

P

B,C

P (D|B, C) P (E|B, C) P (B, C)

Figure 8.3.: Partial computation tree

Conditioning on B allows us to calculate the root of the partial compu- tation tree P (D, E).

P (D, E) = X

B=b

P (D|b, C)P (E|b, C)P (b, C)

We will assume that all variables are discrete, with states F alse and T rue, and the network has the distributions P (A), P (B|A), P (C|A), P (D|B, C), P (E|B, C), P (F |D, E) and P (G|F ) shown in tables (8.1, 8.2, 8.3, 8.4, 8.5, 8.6, 8.7) respectively. Although not all distributions are required for this example, we have included them for completeness.

Table 8.1.: P (A) A = False A = True 0.2 0.8 Table 8.2.: P (B|A) A B = False B = True False 0.25 0.75 True 0.1 0.9

The leaf distribution P (B, C) = P

AP (A)P (B|A)P (C|A) shown in ta-

ble (8.8) was calculated in the standard way.

As shown in the computation tree we need to calculate:

P (D, E) =X

B,C

P (D|B, C)P (E|B, C)P (B, C)

As we are conditioning on B this becomes:

P (D, E) =X

C

P (D|B = F alse, C)P (E|B = F alse, C)P (B = F alse, C)

+X

C

P (D|B = T rue, C)P (E|B = T rue, C)P (B = T rue, C)

(8.2)

Tables (8.9,8.10) show the result of these calculations. In this example we are conditioning on a single variable, but we could have conditioned on additional variables if required.

We then marginalize out C, as shown in tables (8.11,8.12).

Finally, we simply take the sum of table (8.11) and table (8.12), to get the desired result P (D, E) as shown in table (8.13).

Table 8.3.: P (C|A) A C = False C = True False 0.3 0.7 True 0.4 0.6 Table 8.4.: P (D|B, C) B C D = False D = True False False 0.15 0.85 False True 0.35 0.65 True False 0.22 0.78 True True 0.91 0.09

8.5. Conclusion

We have explored a number of scenarios in which limited memory inference is increasingly important, and presented a new technique called Context Conditioning, which can be used to extend existing algorithms, in order to increase scalability and/or limit memory consumption. New algorithms do not need to be written from scratch to benefit, and any code developed can be shared between more than one inference algorithm.

8.5.1. Further work

Further investigation is required to quantify the advantages of this approach under a variety of memory conditions. In addition, the algorithm which is required to marginalize multiple tables under low memory, may benefit from techniques introduced in Chapter (5).

Table 8.5.: P (E|B, C) B C E = False E = True False False 0.55 0.45 False True 0.32 0.68 True False 0.99 0.01 True True 0.7 0.3 Table 8.6.: P (F |D, E) D E F = False F = True False False 0.67 0.33 False True 0.6 0.4 True False 0.8 0.2 True True 0.05 0.95 Table 8.7.: P (G|F ) F G = False G = True False 0.04 0.96 True 0.19 0.81 Table 8.8.: P (B, C) B C = False C = True False 0.047 0.083 True 0.333 0.537

Table 8.9.: P (C, D, E|B = F alse)

C D E = False E = True False False 0.0038775 0.0031725 False True 0.0219725 0.0179775 True False 0.009296 0.019754 True True 0.017264 0.036686

Table 8.10.: P (C, D, E|B = T rue) C D E = False E = True False False 0.0725274 0.0007326 False True 0.2571426 0.0025974 True False 0.342069 0.146601 True True 0.033831 0.014499 Table 8.11.: P (D, E, B = F alse) D E = False E = True False 0.0131735 0.0229265 True 0.0392365 0.0546635 Table 8.12.: P (D, E, B = T rue) D E = False E = True False 0.4145964 0.1473336 True 0.2909736 0.0170964 Table 8.13.: P (D, E) D E = False E = True False 0.4277699 0.1702601 True 0.3302101 0.0717599

9. Topological inference

9.1. Introduction

The Bayesian network inference technique we introduce here has the same attractive properties as the well known poly-tree algorithm [Kim and Pearl, 1983]. However it is capable of inference on a wider class of models, and to our knowledge has not yet been published. It is particularly useful for absorbing nuisance graphs, evidence graphs in relevance reduced networks, and calculating conflict measures.

For subsequent notation, please refer to section (1.3).

9.1.1. Topological sort

A topological sort of a Bayesian network is a partial ordering of the variables, such that all variables are preceded by their parents.

All Bayesian networks have at least one topological sort, as they are Directed Acyclic Graphs (DAG) and therefore do not allow directed cycles. The pseudo-code in Algorithm 16 efficiently finds a topological sort in a Bayesian network, or indeed any Directed Acyclic Graph (DAG), with complexity O(|V | + |E|) where |V | is the number of nodes, and |E| is the number of links.

9.2. Topological inference

Next we present a technique which we call Topological Inference. Consider the case where there is no evidence (e = ∅). Each variable A in a Bayesian network has an associated conditional distribution P (A|pa(A)) where pa(A) denotes the parents of A and can be empty. If we know P (pa(A)) we can calculate the joint probability as follows:

Algorithm 16: Topological sort Input: dag (Bayesian network) sort ← empty list;

parentCounts ← zeros; Q ← empty queue;

for n=0 ; n < dag.N odes.Count ; n++ do node = dag.N odes[n];

parentCounts[n] = node.LinksIn.Count; if parentCounts[n] = 0 then

Add node to Q; current = 0;

while Q 6= ∅ do

node ← dequeue from Q; Add node to sort;

foreach link in node.LinksOut do nodeChild = link.To;

childIndex = nodeChild.Index; decrement parentCounts[childIndex]; if parentCounts[childIndex] = 0 then

Add nodeChild to Q; if current 6= dag.N odes.Count then

// error - invalid DAG return sort;

From this we can calculate the marginal probability of A:

P (A) = X

pa(A)

P (A, pa(A))

If the variables pa(A) are conditionally independent of each other (D- separated), then P (pa(A)) equals the product of the parent marginals:

P (A, pa(A)) = P (A|pa(A))Y

j

P (pa(A)j)

If we consider each variable from a topological sort in turn, the required parent marginals are guaranteed to have already been calculated, subject to the required conditional independence assumptions.