Junction trees

The elimination sets satisfy the running intersection property (RIP), which is defined as follows. An or- dered sequence of sets C1, . . . , Ck is said to satisfy RIP if, for all 1 < j ≤ k, there is an i < j s.t.

Cj_∩(C1∪ · · · ∪Cj−1)⊆Ci, e.g.,C4∩{C1, C2, C3}={A, B, D}∩{A, B, C} ⊆ C2. Figure B.9 shows an example of a sequence of sets that does not satisfy RIP.

Any sequence of setsC1, . . . , Ck that satisfies RIP can be converted into a junction or join tree (jtree), whose nodes are theCi’s. A tree is said to be a junction tree, or to have the junction tree property, ifC1∩C2 is contained in everyCi (i ₆= 1,2) on the unique path betweenC1andC2, for every pairC1, C2. In other words, all the sets containing someXj form a connected tree: along any path,Xjcannot disappear and then reappear later.

We can construct a jtree from a sequence of sets which satisfy RIP by connectingCj to anyCi, for

i∈ {1, . . . , j−1}, s.t. Cj∩(C1∪ · · · ∪Cj−1)⊆Ci. Let us call this procedurejtree-from-RIP-clqs (algorithm 4.8 in [CDLS99, p55]; see also [Pea88, p113]). Note that this way of constructing a jtree is not valid unless the order of the sets satisfies RIP.

The elimination tree is a jtree of sets, but is not a jtree of maximal cliques. It is conventional to remove the redundant sets.4 We can do this by a simple pruning procedure: when we createCj, we check if there is any higher numbered (already created) clique,Cj _⊂Cifori > j, and if so, we simply omitCj from the output sequence. Unfortunately, the resulting sequence may no longer satisfy RIP. For example, consider the elimination tree in Figure B.10, with sequence of elimination setsC1 = {1},C2 = {1,2},C3 = {2,3},

C4={3,4}andC5={2,3,5}. If we deleteC3, we violate RIP.

There is a way to remove redundant sets and keep RIP which we shall callRIP-prune: see Lemma 4.16 of [CDLS99, p60] for details. Alternatively, we can use max cardinality search to find a perfect elimination ordering, and then use Algorithm 4.11 of [CDLS99, p56] (which we callRIP-max-clqs-from-MCS) to find the maximal cliques of the chordal graph, in an order that satisfies RIP. Either way, once we have a sequence of maximal cliques that satisfies RIP, we can usejtree-from-RIP-clqsto create a jtree of maximal cliques. For example, see the jtree Figure B.12; compare this with the etree in Figure B.8. In general, a jtree may be much smaller than an etree.

Fortunately, there is a much simpler way to construct a jtree from a set of maximal cliques, even if their ordering does not satisfy RIP. We first create a junction graph, in which we add an edge betweeniandjif

4_{It is not always a good idea to only use maximal cliques. One advantage of an etree is thatC}

j\Cijis always a singleton, where Cijis the intersection between two neighboring cliques. This means that we only have to marginalize one variable at a time during message passing (see Section B.4). When variables can be of mixed types, e.g., random variables and decision variables in an influence diagram, this can result in simpler algorithms: see e.g., [CDLS99, ch.8]. [GLS99] also gives cases where it is better not to require that the cliques be maximal, in the context of constraint satisfaction problems.

moralize

prune

triangulate

clq−sizes

etree−from−esets

etree

DAG

G

greedy

jtree

RIP−prune

jgraph−from−max−clqs

jgraph

max−spanning−tree

max−cardinality−search

sim. annealing

elim−order

GT

perfect−elim−order

elim−sets

jtree−from−RIP−clqs

RIP−max−clqs−from−MCS

max−clqs

RIP−max−clqs

node−sizes

Figure B.11: Summary of the steps involved in converting a DAG to a junction tree. The cost of a node isqi, the number of discrete values it can take on. GT is the triangulated version ofG. We now specify the correspondence between some of the square boxes and algorithms in [CDLS99]. Greedy is Algorithm 4.13. Simulated annealing is another way of trying to compute an optimal elimination ordering; see Sec- tion B.3.5. max-cardinality-searchis Algorithm 4.9. RIP-max-clqs-from-MCSis Algorithm

4.11. RIP-pruneis Lemma 4.16. jtree-from-RIP-clqsis Algorithm 4.8. etree-from-esets

is Algorithm 4.14. See text for details.

BCF

ABC

ABD FG

Figure B.12: A jtree for Figure B.1 constructed using the elimination ordering(A, B, C, D, F, G). Compare with the etree in Figure B.8.

Ci∩Cj 6=∅. We set the weight of this edge to be_|Ci∩Cj|, i.e., the number of variables they have in common. Any maximal weight spanning tree (MST) of the jgraph is a junction tree (see [JJ94a] or [AM00] for a proof). We can construct an MST using Kruskal’s algorithm inO(|E|log|E|)time, or using Prim’s algorithm in

O(N2)time [CLR90]. (BNT uses Prim’s algorithm, because Kruskal’s requires a priority queue, which is inefficient in Matlab.) The best MST is the one which minimizes the sum of the costs on each edge, where the cost of an edgee= (v, w)is defined asχ(e) =qv+qq₋qv∩w.qvis the cost of nodevin the jgraph, and

is defined asqv=Q_i∈vqi, whereqiis the number of discrete valuesXican take on. If we useχ(e)to break ties when constructing the MST, the result will be an optimal junction tree (for a fixed elimination ordering) [JJ94a].

The overall procedure for converting a DAG to a jtree is summarized in Figure B.11.