Introduction

As with Gaussian processes, an in-depth description of Bayesian belief networks can be found in many other sources (Murphy, 2012; Korb and Nicholson, 2003; Koller and Friedman, 2009). Here we give a short introduction to them, followed by a brief description of the concepts of d-separation and inference using Bayesian networks which will be of importance during the research in Chapter 5.

In purely mathematical terms, a Bayesian network is a probabilistic graphical model in the form of a directed acyclic graph (DAG) representing a particular factorisation of the joint probability of the system (Figure 2.3). Essentially each set of in-going links in the network going in to a particular node represents a factor in the joint distribution that is the marginal conditional distribution of the variable that node represents. The distribution is conditioned upon the nodes that those links are out- going from. These nodes are referred to as the ‘parents’ of the node of interest, and the node of interest is a ‘child’ of those nodes. The joint distribution of the system is then given by

PrpXq “

ź

i

PrpXi |papXiqq (2.17)

whereX is the set of variables defining our system,Xiis one of these variables, and

papXiq are the set of parents of that variable.

Within public health, Bayesian networks are often used to give a causal represen- tation of a system (or at the very least a representation of a set of causal beliefs about the system often obtained from experts on the particular system). We define

(a)

(b)

Figure 2.3: Graphical representations of specific factorisations of the joint dis- tribution over the variables A, B, C, D, E, and F. (a) PrpA, B, C, D, Eq “

PrpE |DqPrpD|CqPrpC|BqPrpB |AqPrpAq. (b) PrpA, B, C, D, E, Fq “

PrpF |EqPrpE |C, DqPrpC|BqPrpD|BqPrpB |AqPrpAq.

our system by a set of variables, and then a link going from variableA to variable

B implies thatAcausesB (as well as thatB is directly probabilistically dependent on A). For example, consider we have a small dog called Merlin. As Merlin is a greedy dog, sometimes when he is taken out on a walk he will eat something off the floor that is bad for him. This can potentially cause him to vomit later. There is always some chance that he may not be walked on a particular day, and even if he is walked he may not find anything to eat. If he is not walked, he may find something accidentally left on the kitchen floor to eat that could also be bad for him. Potentially he may be ill, which could also lead to him vomiting despite not eating anything bad for him. If he is ill then he may also show other symptoms. These relationships are encoded in a Bayesian network shown in Figure 2.4.

As we have described, each of the possible states of each variable are inherently probabilistic in their possibility. We encode these probabilities as conditional prob- ability tables (CPTs) for each variable. Each entry in a table states the probability of a given state of that variable given the particular states its parents are inhabit- ing. Which variables are linked to which other variables forms the structure of our model, and the conditional probability tables form the parameters of our model, i.e.

θ_ijk “PrpXi “j|papXiq “kq . (2.18)

Note that for Bayesian network model parameters we signify nodes by lower indices and states (or node and parent state combinations) by upper indices.

W

A

V

I

S

Figure 2.4: Bayesian belief network for the “Merlin the dog” example. Each of the variables has two states, yes (Y) or no (N). W - Merlin was walked. A - Merlin ate something bad for him. I - Merlin is ill. V - Merlin has vomited. S - Merlin has exhibited other illness symptoms (e.g. temperature, diarrhoea etc). Each node is accompanied by their conditional probability table, showing the likelihood of each state dependent on the parents of that node (if any).

(a)

(b)

Figure 2.5: Examples of d-separation relationships. (a) A K G || tB, Cu. (b)

AKG|| tB, Fu.

such as “if we can prevent him from eating something he should not, how much can we reduce the likelihood of him making a mess of our floors?” and “if Merlin is ill and vomits, what is the likelihood that he has also eaten something he should not?”. These types of queries rely on concepts of conditional independence which we describe next.