Decision networks

In order to make good decision under uncertainty, two factors need to be known; i) the likelihood of every possible outcome, and ii) the preferences of the decision-maker with regard to each of the outcomes. Bayesian networks provide a sound methodology to obtain the probability of outcomes as discussed in the previous sections. Combining Bayesian networks with preferences will give a powerful foundation of making decisions under uncertainty [21,p. 89]. As discussed in section 1.3, preferences are better expressed in terms of a utility function the maps an outcome to a numerical value that conveys a useful aspect of the outcome to the decision-maker.

Once the utility function is defined, the expected utility of each decision is calculated by [21,p. 89]:

| ∑ | , | (70)

where is i-th the possible outcome, A is the actions for outcome , | is the expected utility of each outcome when action A is made and | , is the conditional probability of the i-th outcome giving the current evidences E and action A is made. The action with the highest utility is often selected if the principle of maximum expected utility is followed [21,p. 90]. The principle of maximum expected utility states that rational agents have a tendency to prefer the action that results in the maximum possible utility [21,p.

90]. Decision networks may be expressed graphically by extending BN graph with decisions and utility nodes [21,p. 91]. Decisions, or actions, are often

Bayesian Artificial Intelligence 71

represented with a rectangular shape and the utility nodes with diamond shapes.

For example, the BN for monitoring an ICU patient given in figure 10 can be extended by considering what decisions a nurse would make for every possible state of the patient and the utility of each decision. Since there are only two possible states for the patient: alive and deteriorating then the nurse may only make one decision which is to contact the doctor in case the patient is deteriorating and to continue monitoring otherwise. The expected utility of contacting the doctor has an effect on the next state of the patient so a second node should be added to simulate temporal relationship between the current state of the patient, next state and the undertaken action. The utility function itself is used to map the decision of contacting the doctor to a numerical value that reflects whether the decision led to the recovery of the patient or further deterioration, see figure 13.

S

B O

H

A

S1

U

Figure 13. A simple decision network based on the DBN of figure 10

Bayesian Artificial Intelligence 72

The decision block A belongs to the class of actions known as intervening actions [21,p. 97]. Intervening actions are those actions that have an effect on the probability of the outcome of the network. In the case of figure 13, making a decision to call the doctor would change the likelihood of the current state of the patient. Non-intervening actions are those which do not affect the probability of the system for example, betting in a gambling game [21,p. 97]. Although the decision network of figure 13 is very simple, it can be extended to include more than one decision in a sequential decision-making fashion such as to approximate what decision-maker would do in a course of actions. For example, the nurse may decide to make some test before deciding to contact the doctor to further confirm that the patient is really deteriorating. Such type of actions are referred to as test nodes [21,p. 98].

However, by the time the test is performed, it may be too late for the patient so a test node should be accompanied by a cost node. Similarly to the expected utility node, the cost node maps a cost-wise aspect of performing a test into a numerical value [21,p. 98]. A test has an effect on making further decision and can be regarded as evidences but it has no effect on the states of the process.

Figure 14 shows a simple addition of a test node (T) with cost (C) that a nurse can undertake to confirm the readings of the ICU monitors.

The dashed arrow between the test node (T) and the action node (A) shows which one should be performed first and is known as the precedence link [21,p. 98]. In order to evaluate the utility of each decision, the decision network is transferred into a decision tree model [21,p. 101]. Each possible outcome of an action or test is represented by a branch starting with the

Bayesian Artificial Intelligence 73

action/test that has the highest precedence and continue to divide each branch according to the possible actions/tests in the sequence of actions/tests, then the tree is further branched based on the possible outcomes of the states nodes and finally each ending leaf is weighted by its

expected utility. Once the decision tree diagram is plotted, the calculation of the expected utility of actions follows from the bottom leaves where the utility nodes reside to the action/test node of the highest precedence by multiplying the value of the expected utility by its likelihood and then summing over the next branch until the root is reached [21,p. 103]. Once the utility of each decision is estimated, the one with the highest utility is selected if the principle of maximum expected utility is followed. Although analyzing the decision network through a decision tree seems appealing from simplicity point of view, it is computationally inefficient as it involves repetition of similar mathematical terms. It can be improved using the same techniques introduced in the

S

B O

H

A

S1

U T

C

Figure 14. The addition of a test node to the network of figure 13

Bayesian Artificial Intelligence 74

previous section for Bayesian network inference such as structure transformation and variable elimination [21,p. 104]. Decision nodes can also be added to a DBN so as to model the temporal evolution of actions through time and thereby creating a dynamic decision network (DDN). Figure 15 shows how the decision network for monitoring an ICU patient of figure 13 can be combined with the DBN of figure 12.

The DDN of figure 15 assumes that the sequence of actions (shown as A1, A2 and An) starts after the arrival of the first evidence and that they have precedence from left to right as indicated by the dashed arrow. In addition, it assumes that the decision-maker is interested in the utility of making the first (n) sequence of evidences which is modelled by the inclusion of only one

Figure 15. An example of DDN based on figure 12

Bayesian Artificial Intelligence 75

each slice, then the measured utility will be the change in utility from between the previous and the current action [21,p. 110].