Dynamically consistent updating of multiple prior beliefs – An algorithmic approach

(1)

Dynamically consistent updating of multiple prior beliefs – An

algorithmic approach

Eran Hanany

a,⇑

, Peter Klibanoff

b

, Erez Marom

a a

Department of Industrial Engineering, Tel Aviv University, Tel Aviv 69978, Israel b

Department of Managerial Economics and Decision Sciences, Kellogg School of Management, Northwestern University, Evanston, IL 60208, USA

a r t i c l e

i n f o

Article history:

Received 18 January 2011

Received in revised form 28 June 2011 Accepted 29 June 2011

Available online 12 July 2011 Keywords: Utility theory Uncertainty modeling Risk analysis Updating beliefs Ambiguity

a b s t r a c t

This paper develops algorithms for dynamically consistent updating of ambiguous beliefs in the maxmin expected utility model of decision making under ambiguity. Dynamic con-sistency is the requirement that ex-ante contingent choices are respected by updated pref-erences. Such updating, in this context, implies dependence on the feasible set of payoff vectors available in the problem and/or on an ex-ante optimal act for the problem. Despite this complication, the algorithms are formulated concisely and are easy to implement, thus making dynamically consistent updating operational in the presence of ambiguity.

1. Introduction

A central task facing any theory of decision making under uncertainty is updating preferences in response to new infor-mation (see e.g.,[35]). Since updated preferences govern future choices, it is important to know how they relate to informa-tion contingent choices made ex-ante. Dynamic consistency is the requirement that ex-ante contingent choices are respected by updated preferences (see[10]for a formal deﬁnition of dynamic consistency and a detailed discussion of its relation to other dynamic consistency concepts in the decision theory literature). This consistency is implicit in the standard way of thinking about a dynamic choice problem as equivalent to a single ex-ante choice to which one is committed, and is thus ubiquitous in decision analysis.

Under subjective expected utility (EU), updating preferences by applying Bayes’ rule to the subjective probability isthe standard way to update. Why is this so? Dynamic consistency is the primary justiﬁcation for Bayesian updating. Not only does Bayesian updating imply dynamic consistency, but, if updating consists of specifying a conditional probability measure for each (non-null) event, dynamic consistency implies these conditional measures must be the Bayesian updates. Even under the view that Bayesian updating should be taken as given, this tells us that dynamic consistency comes ‘‘for free’’ under EU.

The study of dynamic consistency is in a well deﬁned sense the study of optimal updating, as dynamically consistent up-date rules result in maximal (ex-ante) welfare. Moreover, since dynamic consistency leads to a well-established theory of updating under expected utility, it makes sense to ask what it implies for the updating of more general preferences. In a re-cent paper[10], Hanany and Klibanoff pursued this strategy to update preferences of the max–min expected utility (MEU) form[6]. For these preferences, beliefs are ambiguous in the sense that they are represented by a (compact and convex) set of

⇑ Corresponding author.

E-mail addresses:[email protected](E. Hanany),[email protected](P. Klibanoff),[email protected](E. Marom).

Contents lists available atSciVerse ScienceDirect

International Journal of Approximate Reasoning

(2)

probability measures, rather than the usual single measure of EU, and acts are evaluated by the minimum expected utility generated by these measures. MEU preferences are widely used in modeling ambiguity averse behavior, as exempliﬁed by the famous[3]paradoxes (for a survey of economic applications of MEU see[20]). The objective of this paper is to offer ways to explicitly compute the decision maker’s (DM’s) updated beliefs according to the update rules suggested by Hanany and Klibanoff[10], thus making these rules operational. For each rule, the updated ambiguous beliefs are computed using con-structive algorithms. The algorithms we describe allow explicit implementation of these rules, for example via computer programs. The rules allow the dynamically consistent updating of any set of MEU beliefs upon observing any non-null event. These rules all are generalizations of Bayesian updating in the sense that they specialize to Bayes’ rule when the set of mea-sures is a singleton. In common with Bayesian updating and many other models, our approach does not address preferences conditional on completely unanticipated (i.e., null) events. Similarly, it does not consider costs, cognitive or otherwise, of describing events. For a discussion of these and other aspects of dynamic decisions see, e.g.,[9,19,25].

To better understand the issues involved in dynamically consistent updating under ambiguity, consider a version of Ellsberg’s three-color problem. There is an urn containing 120 balls, 40 of which are known to be black (B) and 80 of which are somehow divided between red (R) and yellow (Y), with no further information on the distribution. A ball is to be drawn at random from the urn, and the DM faces a choice among bets paying off depending on the color of the drawn ball. Any such bet may be written as a triple ðuB;uR;uYÞ 2R3 where each ordinate represents the payoff if the respective color is drawn. Typical preferences have (1, 0, 0) preferred to (0, 1, 0) and (0, 1, 1) preferred to (1, 0, 1), reflecting a preference for the less ambiguous bets. Notice that these preferences entail a preference to bet on black over red when no bet is made on yellow and a preference to bet on red over black when a bet is also made on yellow (thus violating the sure-thing principle of EU). Now consider a simple dynamic version of this problem. In the dynamic version, there is an interim information stage, where the DM is told whether or not the drawn ball was yellow. The DM is allowed to condi-tion her choice of betting on black or red on this informacondi-tion. In the first choice pair, the bet on black or red is not paired with a bet on yellow, so the choice ‘‘Bet onB’’ leads to the payoff vector (1, 0, 0) while the choice ‘‘Bet onR’’ leads to pay-offs (0, 1, 0). In the second choice pair, a bet on yellow is included, so the choice ‘‘Bet on B’’ leads to the payoff vector (1, 0, 1) while the choice ‘‘Bet onR’’ leads to payoffs (0, 1, 1). Since the DM can condition on whether or not a yellow ball is drawn, the complete set of pure strategies available in each choice problem is: (‘‘Bet onB’’ if not Y; ‘‘Bet onB’’ if Y), (‘‘Bet onB’’ if not Y; ‘‘Bet onR’’ if Y), (‘‘Bet onR’’ if not Y; ‘‘Bet onB’’ if Y), (‘‘Bet onR’’ if not Y; ‘‘Bet onR’’ if Y). Notice that when translated into payoff vectors, these strategies yield only {(1, 0, 0), (0, 1, 0)} in the first choice problem and {(1, 0, 1), (0, 1, 1)} in the second choice problem. If payoff vectors are what the DM cares about, then this provides a strong argument that choices in the dynamic version of the Ellsberg problem should be the same as in the original problem since the feasible payoff vectors are the same. This is closely related to ensuring that the value of information is always non-negative, an-other desirable principle for decision making (see, e.g.,[30]). If choices in the dynamic version of the problem differed from those in the original problem (where the choices are made before the information is revealed) then the DM would strictly prefer to face the original problem, thus declining the (free) information.

What kind of rules for updating ambiguous beliefs imply that the choices in the dynamic version are the same as in the original problem (not only for this example, but in general)? In[10]we provide such rules, and also show that any such rules must depend on the feasible set of payoff vectors available in the problem and/or on an ex-ante optimal act for the problem. To make this clear, consider the following speciﬁcation of MEU preferences that allow the Ellsberg choices in the example above. For any MEU preference over payoff vectors in R3, there exists a compact and convex set of probability measures,C, over the three colors and a utility function,u:R!R, such that8f; g2R3_; _f_%_h ₍₎_min

q2CRðufÞdqP minq2CRðuhÞdq. Letu(x) =xfor allx2Rand letC¼ 13;

a

;23

a

j

a

2 1 4;125

, a set of measures symmetric with respect to the probabilities of red and yellow and consistent with the information that 40 of the 120 balls are black. Observe that, indeed, (1, 0, 0) is preferred to (0, 1, 0) and (0, 1, 1) is preferred to (1, 0, 1) according to these preferences. If we apply full Bayesian updating (Bayesian conditioning of each measure inC) conditional on the eventE= {B,R}, the updated set of mea-sures isCE¼ ð

a

;1

a

;0Þj

a

2 4

9;47

. According to these updated preferences, ‘‘Bet onB’’ is strictly preferred to ‘‘Bet onR’’ conditional on learningE= {B,R}, leading (1, 0, 0) to be selected over (0, 1, 0) in choice problem 1, in agreement with the unconditional preferences, but also leading (1, 0, 1) to be selected over (0, 1, 1) in choice problem 2, in conﬂict with the unconditional preferences. It follows that for an update rule to maintain the choices according to the unconditional pref-erences for both choice problems, the rule must depend on the feasible set and/or on an ex-ante optimal act. A dynam-ically consistent update in this example corresponds to Bayesian updating of measures in a particular strict subset of the ex-ante set of measures, speciﬁcally the conditional set of measures isC1_E¼ ð

a

;1

a

;0Þj

a

2 1

2;47

for choice problem 1 andC2E¼fð

a

;1

a

;0Þj

a

2 49;12

gin choice problem 2. This emphasizes that although natural analogues to updating beliefs under EU exist for updating beliefs under ambiguity, dynamic consistency requires novel procedures that operate some-what differently.

The prior literature on updating ambiguous beliefs in MEU proposes and explores rules that, in fact, fail dynamic consis-tency for at least some MEU preferences. This literature includes many well-known rules, such as full (or generalized) Bayes-ian updating, maximum likelihood updating, or Dempster–Shafer updating. Full BayesBayes-ian updating calls for updating each measure in a set of priors according to Bayes’ rule (see[13,14,5,34,32,24,23,28,33,4], for papers suggesting, deriving or char-acterizing this update rule in various settings). Maximum likelihood updating says that, of the set of priors, only those mea-sures assigning the highest probability to the observed event should be updated using Bayes’ rule, and the other priors should be discarded (see[7]). Kriegler[17]advocates a hybrid approach applying full Bayesian updating to a set of measures

(3)

formed through

e

-contamination where additionally

e

is updated through a maximum likelihood procedure. Both full Bayesian and maximum likelihood updating are given an interpretation in terms of epistemic belief hierarchies by Walliser and Zwirn[31]. For Dempster–Shafer updating see[2,27]. Jaffray[15]suggests that the inconsistency between unconditional and conditional preferences might be resolved in a way that is a compromise between the different preferences. He examines a selection criterion that chooses a plan that is ‘‘not too bad’’ in a utility sense according to any of these preferences and is not dominated in that no feasible plan is better according to all the preferences. Nielsen and Jaffray[22]construct algorithms for implementing the approach suggested by Jaffray[15]in the context of risk.

In contrast to this literature, Hanany and Klibanoff[10]identify update rules that are dynamically consistent for any MEU preferences upon observing any non-null event. Given the necessary dependence of the consistent rules on the feasible set and/or an ex-ante optimal act, the practicality of their implementation is an issue. This motivates the present paper, where we provide algorithms for computing the updated beliefs determined by these consistent rules. In developing the algorithms and proving that they implement the various rules, we draw on techniques from convex analysis.

In the next section we describe the framework for our analysis. Section3describes algorithms to compute updated be-liefs. Section4provides a short summary. Proofs are collected inAppendix A.Appendix Bcollects some useful results from polyhedral theory. Code for an implementation of the algorithms in the paper using WolframMathematicais available as an

Online Supplementon the journal webpage or from the authors.

2. Framework

Consider the setXof all simple (i.e., finite-support) lotteries over a set of consequencesZ, a finite set of states of natureS endowed with the algebraRof all events, and the setAof all acts, i.e. functionsf:S?X. Consider a non-degenerate max– min expected utility (MEU,[6]) preference relation%_over_A_{, for which there exists a compact and convex set of probability} measures with a finite set of extreme points,C, and a vonNeumann–Morgenstern (vN–M) EU function,u:X!R, such that

8f; h2 A; f%_h ₍₎ _min_q₂_CR_ð_u_f_Þ_dq_P_min_q₂_CR_ð_u_h_Þ_dq_{. Let}_PMEU_{denote the set of all such preference relations. If}%_is non-degenerate,Cis unique anduis unique (among vN–M EU functions) up to positive afﬁne transformations. As usual,

anddenote the symmetric and asymmetric parts of%.

LetN ð%_Þ_{denote the set of events}_E₂_R_{for which}_"_q₂_C_,_q₍_E_{) > 0. We limit attention to updating on events that are} non-null in this sense. ForE2R, letD(E) denote the set of all probability measures onRgiving weight 0 toEc_{. For any}_q₂_D(_S₎ withq(E) > 0, we denote byqE2D(E) the measure obtained through Bayesian conditioning ofqonE.

LetBdenote the set of all non-empty subsets of actsB#Asuch thatBis convex (with respect to the usual[1]mixtures) and compact with a ﬁnite set of extreme points. Elements ofBare considered feasible sets and their convexity could be jus-tiﬁed, for example, by randomization over acts. Compactness is needed to ensure the existence of optimal acts.

Assume a preference%_{2 P}MEU_{, an event}_E_{2 N ð}%_Þ_{and an act}_g_{2 A}_{chosen according to}%_{from a feasible set}_B_{2 B}_before the realization ofE(i.e.,g%_f_{, for all}_f₂_B_{). Denote by}_T _{the set of all such quadruples}_ð%_;_E_;_g_;_B_Þ_{. An update rule is a function}

U:T ! PMEU_{, producing for each}_ð_%_;_E_;_g_;_B_{Þ 2 T} _{a MEU conditional preference, denoted}_%

E;g;B, representable using the same (up to normalization) vN–M utility function uas % _{and a non-empty, closed and convex set of conditional measures}

CE,g,B# CE{qEjq2C}, with a ﬁnite set of extreme points. Such a conditional preference is viewed as governing choice upon the realization of the conditioning eventE. LetUBayes_{denote the set of all such update rules.}

Abusing notation in the standard way,x2Xis also used to denote the constant act for which"s2S,f(s) =x. For any f; h2 A, we usefEhto denote the act equal tofonEandhonEc_{, the complement of}_E_{. General vectors in}_RjSj _{will be} called utility acts. Ifaandbare utility acts, we useaEbto denote the utility act equal toaon EandbonEc. SinceSis ﬁnite, we sometimes identify probability measures with vectors inRjSj _{normalized to sum to 1. For an arbitrary convex,} compact set of real vectors,A, denote by ext(A) the set of extreme points of A. Let C¼extðCÞ and B¼extðuBÞ, thus C¼coðCÞanduB¼coðBÞ, wherecodenotes the convex hull operator. This implies thatCEis the convex hull of a ﬁnite number of points. For each a2RjSj _and _n₂_R_{, denote the half-space} _f_c₂_RjSj_j_a_c_P_n_g _by _Wn

a and the hyperplane fc2RjSj_j_a_c_¼_n_g_by_Hn

a.

As discussed in the introduction, the update rules considered in this paper depend on an initially optimal actgand the feasible setB. Before introducing update rules, we describe a simple algorithm for computing an initially optimal act.

Algorithm 2.1. Solve the linear program,

o_;_k ð Þ 2arg max ðo;kÞ2R ½0;1jBj o; s:t: P s2S P a2B kaasqsPo;

8

q2C; P a2B ka¼1: ðPÞ

(4)

Proposition 2.1. The act g is optimal in B according to%_.

The following example will be used throughout the paper to demonstrate our results.

Example 2.1. Consider a state spaceS= {1, 2, 3, 4} and two investment optionsl,nleading to state contingent monetary payments (5,5, 20, 20) and (10,15, 10,15), respectively. For simplicity, assume risk neutrality and take utilityuover monetary outcomes inZ¼Rto be the identity function (our analysis applies to any assumption on risk attitudes). The ambiguous beliefs on the state space are represented by the setC=co{(0.4, 0.4, 0.1, 0.1), (0.1, 0.4, 0.25, 0.25), (0.4, 0.1, 0.25, 0.25)}. Before investing, it is possible to pay a cost of 1 to reveal whether the eventE= {1, 2} is true and make an investment decision based on this information.Fig. 2.1illustrates the setsCandD(E) within the simplex of all probability measures over S. Thus, before randomization, there are seven feasible alternatives: the two investment options l,nwithout buying the information, the non-investment option and four information contingent investment possibilities. This is summarized by the feasible utility set uB=co{(5,5, 20, 20), (10,15, 10,15), (0, 0, 0, 0), (6,6, 19, 19), (9,16, 9,16), (6,6, 9,16), (9,16, 19, 19)}. ApplyingAlgorithm 2.1results in an optimumo⁄_{= 1 and}_k⁄_{= (0, 0, 0, 0, 0, 0, 1), i.e. buy the information and} invest innifEand inlif notE. Note thatb=ug= (9,16, 19, 19), evaluated by minq2CPs2Sbsqs¼1. In the next section we analyze the updated ambiguous beliefs in this problem. Throughout the paper we analyze the case whereEis observed – similar analysis, omitted for brevity, can be done for the case whereEc_{is observed.}

3. Computing updated beliefs

3.1. Dynamically consistent update rules

Dynamic consistency, an important and intuitive property of update rules, means that initial contingent plans should be respected after receiving information. In our framework, the actgshould remain optimal also conditionally.

Axiom 3.1. DC(Dynamic consistency). For anyð%_;_E_;_g_;_B_{Þ 2 T}_,_{if f}₂_{B with f}₌_g_on_Ec_{, then}_g%_E

;g;Bf.

Observe that conditional optimality ofgis checked against all feasible actsfsuch thatf=gonEc_{. Why check conditional} optimality only against these acts? Consider an environment where the DM has a ﬁxed budget to allocate across bets on var-ious events. It would be nonsensical (and would violate payoff dominance on the realized event) to require that the ex-ante optimal allocation of bets remained better than placing all of one’s bets on the realized event. This justiﬁes the restriction of the conditional comparisons to acts that could feasibly agree onEc. An actf=gonEcwill be calledcomparabletog.

Among rules satisfying desiderata such as dynamic consistency, there are those that are mostconservativein the sense of maintaining the most ambiguity in the process of updating. In general, this means that the updated set of measures should be the largest possible subset ofCE, in the sense of set inclusion, while still satisfying the desiderata. Examining such rules is particularly illuminating because they reveal the precise extent to which dynamic consistency forces the DM to eliminate measures present in the unconditional set when updating. If, for example, one views full Bayesian updating (updating all measures in the initial set) as ‘‘the right thing to do’’ then examining these rules shows how far one must depart from this to maintain consistency. Note that if dynamic consistency is ignored, and all rules inUBayes_{are considered, ambiguity} max-imization would uniquely select full Bayesian updating.

Let us now observe the implications ofDCfor updating. The following algorithm can be used to compute the set of up-dated beliefs under the unique ambiguity maximizing update rule (denotedUDCmax_{) within the dynamically consistent rules} inUBayes. Having found an initially optimal actgin Step3.1.A, the algorithm computes in Step3.1.Bthe set of feasible utility acts comparable tog. Step3.1.Ccomputes Bayesian conditional measures normal to hyperplanes supporting this set atg.

(5)

Only these measures can be used to evaluategconditionally if dynamic consistency is required. If we were to stop the algorithm at this point and returncoðQEÞ, this would be a dynamically consistent update rule but need not maximize ambi-guity. To maximize ambiguity, Step3.1.Dﬁnds the minimum evaluation ofgaccording to these measures, and computes the updated beliefs as the set of conditional measures that evaluategat least as high as this minimum.

Algorithm 3.1.

Step3.1.A: Computeb=ugando_¼_min

q2C P

s2Sbsqs.1 Step3.1.B: ComputeLext coðBÞ\s2EcHbs

IðsÞ

h i

, whereIðsÞ 2RjSj_{denotes an}_s_{-indicator, i.e.}_Is₍_s_{) = 1 and}_I

^

sðsÞ ¼0 for^s–s.2 Step3.1.C: ComputeQEext coðCEÞ\_a₂_LW0ba

h i

, whereCEextðCEÞ.

Step3.1.D: ComputeUEext coðCEÞ \W0bb11

h i

, whereb₁min_q₂_Q E

P

s2Sbsqsand1denotes the constantð1;. . .;1Þ 2RjSj. ReturncoðUEÞ.

Proposition 3.1. Algorithm3.1results in the updated set of measures, CE,g,B, produced by the update rule UDCmax.

Example 3.1.We apply the algorithm to our leading example. Since we already have an optimal utility actb= (9,16, 19, 19) and its valueo⁄

= 1, we start by ﬁnding the setL, the extreme points of the set of feasible acts comparable tog. The set of extreme points ofcoðBÞ \H19Ið3Þis {(9,16, 19, 19), (6,6, 19, 19), (4.75,4.75, 19, 19)}. Then intersecting withH

19

Ið4Þ leaves this set unchanged, so this isL. Note that this set includes the optimal utility actb, the utility act corresponding to the choice of buying the information and investing always inl, and the utility act (4.75,4.75, 19, 19) resulting from a convex com-bination of the no investment option and investing inlwithout buying the information. Next we find the setCE, the extreme points of the set obtained by Bayesian conditioning of all measures inC(see the top part ofFig. 3.1). The conditionals of the measures inCare {(0.5, 0.5, 0, 0), (0.2, 0.8, 0, 0), (0.8, 0.2, 0, 0)}. Since the first of these conditionals is in the convex hull of the other two,CE¼ fð0:2;0:8;0;0Þ;ð0:8;0:2;0;0Þg. Notice that if one adopted this set as the updated ambiguous beliefs, i.e. followed the full Bayesian rule, the optimal choice givenEwould be not to follow the optimal act and instead invest inl. This would result in the utility act (6,6, 19, 19), which is dominated (in every state) by the feasible act invest inlwithout buying the information. This demonstrates that full Bayesian updating may be an undesirable update rule. In contrast,UDCmax is dynamically consistent, and thus does not suffer from such phenomena. To see this, we continue to follow the algorithm and next findQE, the extreme points of the set of measures supporting the conditional optimality ofg. SincejLj¼3; co CE is intersected thrice, to ensure measures that support the feasible set comparable tog. Sincebis always an element ofLand bb= 0, the intersection based on b never imposes any restriction. Considering b ð6;6;19;19Þ ¼ ð15;10;0;0Þ; W0ð15;10;0;0Þcorresponds to the set of measures such that 15q110q2P0 (see the middle part ofFig. 3.1), so the set of extreme points of coðCEÞ \W0

ð15;10;0;0Þis {(0.4, 0.6, 0, 0), (0.8, 0.2, 0, 0)}. The ﬁnal intersection, with

W0ð13:75;11:25;0;0Þ, requires 13.75q111.25q2P0 and thus shrinks the set of extreme points to {(0.45, 0.55, 0, 0), (0.8, 0.2, 0, 0)}, which isQE(see the bottom part ofFig. 3.1). Note that any conditional measure putting weight lower than 0.45 to state 1 does not support the conditional optimality ofgbecause it becomes worse than (4.75,4.75, 19, 19). Finally we calculate UE, the set of extreme points of the set of updated ambiguous beliefs. Observe that b1= min{(9, -16, 19, 19)(0.45, 0.55, 0, 0), (9,16, 19, 19)(0.8, 0.2, 0, 0)} =4.75. Since q2co CE ; ðbb₁1Þ qP0 if and only if 13.75q111.25q2P0, which is not weaker than the restrictions already imposed, so UEco CE

\W0bb11¼QE. Thus

CE,g,B=co{(0.45, 0.55, 0, 0), (0.8, 0.2, 0, 0)}. Note that given these updated ambiguous beliefs, the optimal act is conditionally equivalent to4.75, which is better than6, the payment if one invests instead inl. Thus dynamic consistency is satisﬁed. The example shows that it is necessary to eliminate some of the measures in the original set, in particular all measures that conditionally give weight lower than 0.45 to state 1. When a plan is made initially to follow the optimal act, the updated ambiguous beliefs represented byCE,g,Bjustify this plan: contingent on learningE, it will be optimal to carry it out.

Note that computing the updated ambiguous beliefs is a hard problem because it involves computing the extreme points of intersections of convex hulls of ﬁnite sets of points, which is known to be hard. If these computations could be done efﬁciently, then all the algorithms in this paper would be polynomial in the size of the problem.

It is also worth noting that in cases where the feasible set has a special structure such that f; f0₂_B₎_fEf0₂_B_{, the} restrictionf=g on Ec _{in the statement of} _DC _{is superﬂuous, thus the computation of Step} _3.1.B _{can be simpliﬁed to} L fa2Bja¼bonEc

g. Such feasible sets arise, for example, whenever one starts from a decision tree with branches 1

Ifgis not known, one can applyAlgorithm 2.1to carry out this step. 2

When computing the extreme points of the intersection of a list of hyperplanes/half-spaces with the convex hull of a ﬁnite set of points, this algorithm and all subsequent ones use standard procedures from polyhedral theory (e.g., as inAppendix B) and thus we do not detail these procedures here.

(6)

corresponding to eventsEandEc_{and derives}_B_{by ﬁrst specifying what is feasible conditional on}_E_{, denoted}_BE_{, and what} is feasible conditional on Ec_, _denoted _BEc

, and then combining the two so that B¼fh2 Ajh¼

fEf0_{for some}_f₂_BE

; f0₂_BEc

g. 3.2. More robust dynamic consistency

It may be desirable to strengthen dynamic consistency, so that all initially optimal acts comparable togshould remain optimal conditional onE.

Axiom 3.2 PFI(Robust dynamic consistency)3_{. For any}_ð_%_;_E_;_g_;_B_{Þ 2 T}_{, if}_f₂_B_with_f₌_g_on_Ec_and_f_g_{, then}_f E,g,Bg.

The following algorithm can be used to compute the set of updated beliefs under the unique ambiguity maximizing up-date rule (denotedUDC\PFImax_{) within the rules in}

UBayes_satisfying_DC_and_PFI_{. As in}_{Algorithm 3.1}_{, Steps}_3.2.A_and_3.2.B_ﬁnd an initially optimal actg, the set of feasible acts comparable togand the set of measures supporting the conditional opti-mality ofg. Step3.2.Ccomputes the measures normal to hyperplanes separating the feasible acts comparable togfrom the acts strictly better thang. Step3.2.Duses one of these measures to compute the set of initially optimal acts comparable tog. Step3.2.Ecomputes the Bayesian conditional measures normal to hyperplanes supporting this set atg. Only these mea-sures can be used to evaluategconditionally if robust dynamic consistency is required. To maximize ambiguity, Step3.2.F ﬁnds the minimum evaluation ofgaccording to these measures, and computes the updated beliefs as the set of conditional measures that evaluate all initially optimal acts at least as high as this minimum.

Fig. 3.1.Illustration ofAlgorithm 3.1for computing the update ruleUDCmax

(seeExample 3.1).

3

This axiom was namedPFI(Preservation of Feasible Optimal Indifference) by Hanany and Klibanoff[10]. We call itrobust dynamic consistencyhere, as upon reﬂection, this seems a more informative name. To maintain continuity with our earlier nomenclature we will nonetheless continue to refer to it by the initials

(7)

Algorithm 3.2

Step3.2.A: Computeb=ugando⁄_{as in Step}_3.1.A_of_{Algorithm 3.1}_.

Step3.2.B: ComputeL; QEas in Steps3.1.Band3.1.CofAlgorithm 3.1, respectively. Step3.2.C: ComputeRext coðCÞ \Hob\a2LW

0

ba

h i

. Step3.2.D: ComputeJext co L \Hqog\_q₂_CW0_q_qg

h i

, whereqg₂_R_. Step3.2.E: ComputeKEext co QE \_a₂_JH0ba

h i

. Step3.2.F: ComputeUEext co CE \_a₂_JW0

ab21

h i

, whereb2minq2KE

P

s2Sbsqs. Returnco UE .

Proposition 3.2. Algorithm3.2results in the updated set of measures, CE,g,B, produced by the update rule UDC\PFImax_.

The following example serves to demonstrate the comparative robustness of updating underPFIandDC, as compared to updating under onlyDC. It also illustrates the algorithm above.

Example 3.2. We apply the algorithm to our leading example. It can be veriﬁed that the optimum found usingAlgorithm 2.1 is unique, thus PFI has no bite. Therefore we modify the example slightly and assume that the cost of buying the information is 2. ApplyingAlgorithm 2.1again, the previous optimal act – buy the information and invest innifEand inlif notE– remains optimal. With the new cost, this is an optimal utility actb= (8,17, 18, 18) with optimal valueo⁄_{= 0. The set of} feasible utility acts comparable tognow has the extreme pointsL¼ fð8;17;18;18Þ;ð7;7;18;18Þ;ð4:5;4:5;18;18Þg. The set of measures supporting the conditional optimality of g now has the extreme points QE¼ fð0:5;0:5;0;0Þ;

ð0:8;0:2;0;0Þg(see the top part ofFig. 3.2). Note that the change inLcaused a change inQE. In particular, any conditional measure putting weight less than 0.5 to state 1 now does not support the conditional optimality ofgbecause it becomes worse than (4.5,4.5, 18, 18). Observe thatcoðCÞ \Ho

b ¼ fð0:4;0:4;0:1;0:1Þg, so R¼ fqgg ¼ fð0:4;0:4;0:1;0:1Þg. Next we ﬁndJ, the extreme points of the initially optimal acts comparable tog. The intersectioncoðLÞ \Hoqg results in the extreme points {(8,17, 18, 18), (4.5,4.5, 18, 18)}. SincejCj¼3, we need 3 more intersections, which leave the set unchanged, so this isJ. Note that the second extreme point of the initially optimal acts results from a convex combination of the no investment option and investing inlwithout buying the information. We move on to computeKE, the extreme points of the set of measures supporting the conditional optimality of all initially optimal acts comparable tog. SincejJj¼2 but the intersection based onb2Jnever imposes any restriction, we intersectQEonce, which leads toKE¼ fð0:5;0:5;0;0Þg(see the top part of Fig. 3.2). Finally, we compute UE, the set of extreme points of the set of updated beliefs. Observe that b2= min{(8,17, 18, 18)(0.5, 0.5, 0, 0)} =4.5. Intersecting coðCEÞ with W0ab21 for each a2J leads to UE¼QE (see the bottom part ofFig. 3.2). Thus the updated ambiguous beliefs under UDC\PFImax

are CE,g,B=co{(0.5, 0.5, 0, 0), (0.8, 0.2, 0, 0)}. ApplyingAlgorithm 3.1to compute the updated ambiguous beliefs underUDCmax_{for this example gives the same set}_C

E,g,B. To see the comparative robustness of updating underPFIandDC, consider the updated ambiguous beliefs given a different

Fig. 3.2.Illustration ofAlgorithm 3.2for computing the update ruleUDC\PFImax

(8)

initially optimal act, corresponding to the utility act (4.5,4.5, 18, 18). UnderUDC\PFImax_{, the updated beliefs will be the} same as above because this rule guarantees that all initially optimal acts inco{(8,17, 18, 18), (4.5,4.5, 18, 18)} remain optimal conditionally. However,UDCmax_{for this case coincides with full Bayesian updating, i.e. results in more conditional} measures in the updated set compared to the case with the former initially optimal act. This additional ambiguity would have affected the conditional optimality of the utility act (8,17, 18, 18), while none of it affects the conditional optimality of (4.5,4.5, 18, 18) due to the constancy onE. Incidentally, forUDCmax_{with the initially optimal act (}_4.5,_{4.5, 18, 18), in} contrast toExample 3.1, Step3.1.Dhas an effect: it expands the updated set fromcoðQEÞ ¼cofð0:2;0:8;0;0Þ;ð0:5;0:5;0;0Þg (see the top part ofFig. 3.2) tocoðUEÞ ¼cofð0:2;0:8;0;0Þ;ð0:8;0:2;0;0Þg.

3.3. Reference dependent updating

Consider an additional condition which, in the presence ofDC, impliesPFIand is stronger thanPFIonly for infeasible acts. Axiom 3.3. RA(g as a reference act). For anyð%;E;g;BÞ 2 T, iff2 Awithf=gonEc_and_f

g, thenf%_E

;g;Bg.

One way of viewingRAis as saying that updating must preserve or increase ambiguity affectinggmore than ambiguity affecting any actfindifferent and comparable tog.4_RA_{has the advantage of simplifying the updated beliefs in the following}

important special case. When the initially optimal actgis constant (in utilities) onEc_{, we can use a simple threshold rule based} on the probability each measure assigns toEto obtain the updated set. It follows that a sufﬁcient condition for the threshold rule to apply occurs whenEc_{is a singleton, i.e., information is learned one state at a time, a not uncommon occurrence.}

The strengthening ofPFItoRAdoes not follow from dynamic consistency considerations, however the condition does not seem unreasonable and has the above-mentioned simpliﬁcation as its main virtue. InRA, the part of the indifference curve throughgagreeing withgonEc_{is picked out for a special role in updating. This part of the indifference curve may be thought} of as the portion wheregis being used as a reference act. The axiom requires thatgoccupy an extremal position in the con-ditional preference relative to the other elements of this part of the (unconcon-ditional) indifference curve (throughg) consisting of acts agreeing withgonEc_.

The following algorithm can be used to compute the set of updated beliefs in this special case under the unique ambiguity maximizing update rule (denotedUDC\RAmax

) within the rules inUBayes_satisfying_DC_and_RA_{. The key new step that provides} the simpliﬁcation to a threshold rule is Step3.3.D. This step selects the measures to update by comparing the weight they give toEwithqg₍_E_{) and updates all measures giving weakly more weight than}_qg₍_E_{) (if}_u_g_>_o⁄_on_Ec_{), weakly less weight (if} ug<o⁄

onEc) or all measures inC(ifug=o⁄ onEc).

Algorithm 3.3

Step3.3.A: Computeb=ugando⁄_{as in Step}_3.1.A_of_{Algorithm 3.1}_.

Step3.3.B: ComputeL; Ras in Step3.1.BofAlgorithm 3.1and Step3.2.CofAlgorithm 3.2, respectively. Step3.3.C: Computeqg_E2arg min_f_q

Ejq2Rg

P

s2EbsðqEÞs. Letqg2Rsuch thatq g

Eis its Bayesian conditional onE. Step3.3.D: ComputeUEext qEjq2coðCÞ \Wa

qg_ð_E_Þ

aP

s2EIðsÞ

, where

a

sign[b(Ec₎_o⁄_{]; return}_co_ð_UE_Þ_.

Example 3.3. We apply this algorithm also to our leading example. Assume again that the cost of buying the information is 2. Recall the optimal act g= (8,17, 18, 18) with optimal valueo⁄_{= 0 and the measure in}_R_{¼ f}_qg_{g ¼ fð}₀_:₄_;₀_:₄_;₀_:₁_;₀_:₁_Þg_, which uniquely separates the feasible acts comparable togfrom the acts preferred tog(seeFig. 3.3). Note thatugis con-stant onEc_{. Since}_sign

ð18o_{Þ ¼}₁_; _Waqg_ð_E_Þ

aP_s₂_EIðsÞis equivalent to the conditionq(E)Pq

g₍_E_{). The unique measure in}_co

ðCÞ satis-fying this condition isqg_{, thus}_UE_{includes only its Bayesian conditional. Therefore the updated beliefs are represented by} CE,g,B= {(0.5, 0.5, 0, 0)}. In general, the updated beliefs are not necessarily reduced to a single measure. For example, consider the optimal actg= (0, 0, 0, 0). Sincegis constant onEc_and_sign_ð₀_o_{Þ ¼}₀_; _UE_¼_CE_{. Thus the updated beliefs coincide with} those obtained by the full Bayesian update rule, i.e.CE,g,B= {(0.2, 0.8, 0, 0), (0.8, 0.2, 0, 0)}.

In fact, the algorithm producing the threshold rule is valid somewhat more generally than suggested above. It works whenever there is no ambiguity about the conditional expectation ofgonEc_{, in the sense that the initially optimal act}_g has the same conditional EU onEc_{according to each measure in}_C_{giving positive probability to}_Ec_.

Proposition 3.3. Assume that uðXÞ ¼RandR_Ecbdq_Ecis the same for all q2C with q(Ec)>0.Algorithm3.3, with b(Ec) replaced by

R

Ecbdq_Ecif there exists q2C with q(Ec)>0 and replaced by o⁄otherwise, results in the updated set of measures, C_E,g,B, produced by

the update rule UDC\RAmax_.

4

An alternative strengthening ofPFIis obtained by replacingf%_E_;_g_;_Bgwithg%_E_;_g_;_BfinRA. However, a small modiﬁcation of the example in the proof of Proposition 13 in[10]can be used to show there is no update rule inUBayes_{satisfying the alternative condition (and in fact, no such rule exists in the larger} family of update rules that allows updating measures outsideC).

(9)

The ruleUDC\RAmax_{may also lead to intermediate updated ambiguous beliefs, i.e. to a strict subset of}_CE_{which is not a} singleton, as demonstrated in the following example.

Example 3.4.LetZ¼R; C¼ 8 24;1024;244;242 ; 5 24;244;1024;245

; B¼ ðf2;1;3;0Þ;ð0;3;3;0ÞgandE= {1, 2}. For the unique initially optimal act, b=ug= (0, 3, 3, 0) and o_¼21

12. Note that uðXÞ ¼R and for all q2C, q(E

c_{) > 0 and} R Ecbdq_Ec¼2. Applying Algorithm 3.3, L¼B and R¼ 8 24;1024;244;242 ; 123;123;124;122

(note that arg minq2CPs2Sbsqs¼C). Then we compute min_f_q

Ejq2Rg

P

s2EbsðqEÞs¼1:5 andqg¼ 123;123;124;122

. Sincesign(2o⁄_{) = 1, we intersect}_co

ðCÞwithq(E)Pqg₍_E_{), resulting in} 8 24; 10 24; 4 24; 2 24 ;₂₄6;₂₄6;₂₄8;10₂₄

, and compute the Bayesian conditionals to ﬁndUE. Thus the updated ambiguous beliefs are represented byCE;g;B¼co 49; 5 9;0;0 ;1₂;1₂;0;0 .

For completeness, inAppendix Awe provide a modiﬁcation of Step3.3.Dthat we show allows computation of the up-dated beliefs underUDC\RAmax_{for the general case. To compute the extreme points of the updated beliefs, this modiﬁcation} considers pairs of extreme points ofCand selects zero, one or two points from their convex hull to update.

4. Summary

This paper develops algorithms for updating ambiguous beliefs in the MEU model of decision making under ambiguity. The update rules all satisfy the desirable property of dynamic consistency as was shown in[10]. Some of the rules also satisfy stronger and more robust consistency requirements as well. The algorithms are formulated concisely and are easy to imple-ment, thus making dynamically consistent updating operational in the presence of ambiguity.

We close by mentioning two possible directions for future research. First, the algorithms in this paper deal only with ﬁ-nitely generated sets of beliefs and feasible acts. The question of whether these algorithms can be used to approximate dynamically consistent updating for arbitrary convex sets of beliefs or feasible acts is left open.

Second, although MEU is a popular theory of decision making with ambiguity aversion, it is far from the only one (see, among many,[12,16,18,21,26,29]). In[11]we expand the approach in[10]to characterize dynamically consistent update rules for a very broad class of ambiguity averse preferences. Algorithmic implementation in these more general settings is left open as well.

Acknowledgements

We thank Yigal Gerchak, Tal Raviv, two anonymous referees, the area editor and Thierry Denoeux, the Editor-in-Chief, for helpful discussions and comments. This research was partially supported by Grant No. 2006264 from the United States-Israel Binational Science Foundation (BSF), Jerusalem, Israel. All responsibility for the contents of the paper lies solely with the authors.

Appendix A. Proofs and auxiliary lemmata

To conserve space in this appendix, whenever we refer to[10], we use the abbreviation HK. Proof of Proposition 2.1.Letq₂_C_satisfyP

s2Sbsqs¼o. For everya2B, there existsf2Bsuch thata=uf, sogexists by convexity of uB. Moreover, g is optimal, otherwise there exists g2B and _k_{2 ½}₀_;₁jBj _{such that} _u

g¼P_a₂_Bkaas and omin_q₂_CPs2Su½gðsÞqs>minq2C P s2Su½gðsÞqs, thuso> P s2S P a2Bk

aasqs¼oin contradiction to the maximality ofo⁄. h

Fig. 3.3.Illustration ofAlgorithm 3.3for computing the update ruleUDC\RAmax

(10)

Proof of Proposition 3.1. HK showed (p. 270, Corollary 1) that UDCmax _{is deﬁned by} _C E,g,B= {qEjq2C and R ðugÞdqEPminp2QE;g;B E R ðugÞdpg, whereQE;g;B

E is the set of Bayesian conditionals onEofQ E,g,B

, the measures inCsupporting the conditional optimality ofg, deﬁned byQE;g;B

fq2CjRðugÞdqEP R

ðufÞdqE for allf2Bwithf=gonE c

}. Since the hyperplanes used in Step3.1.Barec2RjS_j_cs_¼_bs_;_co_ð_L_{Þ ¼}_u_f_f₂_B_j_f_¼_g_on_Ec_{}. By deﬁnition,}_QE;g;B

E is the intersection of CEand all sets of the form q2CEjPs2SðbsasÞqsP0

wherea2u{f2Bjf=gonEc}, thusQE;g;B

E ¼coðQEÞ. The fact that it suffices to apply a finite number of intersections (8a2Land not"a2u{f2Bjf=gonEc}) follows fromProposition A.1 below. Finally, by definition ofUDCmax_,_CE

,g,Bis the set q2CEjPs2Sðbsb1ÞqsP0

. h

The following proposition shows that algorithms for computing the intersection of a compact, convex set with a finite number of half-spaces or hyperplanes (see e.g.Algorithm B.2inAppendix B) can also be used to compute the intersection of such a set with aninfinitenumber of half-spaces/hyperplanes, when the latter are defined by normals taken from a com-pact, convex set having a finite number of extreme points.

Proposition A.1. Fixn2R. Let A;DRjSjbe convex, compact sets, each with a ﬁnite number of extreme points. Denote ext(D) by fak_gr k¼1. Then ext A\ r k¼1 Wn ak ! ¼ext A\ a2D Wn a ! : The same is true when substituting Wn

awith H n a. Proof. LetDn (a) = {c2AjcaPn}. By deﬁnition,AT_a₂_DWn a¼ T a2DD n ðaÞandATr_k_¼1Wn a¼ Tr k¼1D n

ðak_Þ_{, so it remains to show that} T a2DD n ðaÞ ¼Trk¼1D n ðak_Þ_{. Since} r_ak k¼1#D; T a2DD n ðaÞ#Tr_k_¼1Dn

ðak_Þ_{. On the other hand, let}_c₂Tr k¼1D n ðak_Þ_{. Then}_"_k_{= 1,}_{. . .}_,_r_, cak_P_n_{. Let}_a₂_D_{, so}_a_¼Pr k¼1kkakwherekk2½0;1; Pr k¼1kr¼1. Thenca¼c Pr k¼1kkak¼ Pr k¼1kk cak PPr_k_¼1kkn¼n, hencec2Ta2DDn(a), i.e.Trk¼1D n ðak_Þ_#T a2DD n

ðaÞ. The same arguments hold when substitutingWn

awithH

n

aandPwith =. h To show thatAlgorithm 3.2provides the desired updated beliefs, the following notation is useful.

Notation A.1. GivenEandh2B, deﬁne the set of feasible acts comparable and indifferent tohto be

JhB¼ ff2Bjfhandf¼honE c

g:

Proof of Proposition 3.2. HK showed (p.276, Proposition 5) that UDC\PFImax _is _deﬁned _by _CE ;g;B¼ qEjq2Cand R ðufÞdqEPminp2KE;g;B_E R ðugÞdpfor allf2Jg B n o , whereKE;g;B

E is the set of Bayesian conditionals onEofK E,g,B_, the measures supporting the conditional optimality of all acts initially optimal and comparable to g, deﬁned by KE;g;B

q2CjRðugÞdqEP R

ðufÞdqEfor allf2Bwithf¼gonE c and RðugÞdqE¼ R ðufÞdqE if;in addition;fg . We ﬁrst show that uJg B¼fa2uBja¼bon E c_; _min q2CPs2Sasqs¼o¼Ps2Sasq g

s, where qg2QE;g;B\arg minq2CPs2Sbsqs o (see the proof of Proposition 3.1for the deﬁnition of QE,g,B). To see this, note that if ais an element of the r.h.s then a2uJg

B, becausea2u{f2Bjf=gonE

c_{} and min}

q2CPs2Sasqs¼minq2CPs2Sbsqs. For the other direction, supposea2uJgB. Existence ofqg₂_QE;g;B_\_{arg min}

q2CPs2Sbsqs is guaranteed byLemma A.1 of HK stated below. Sincea2uJgB; P s2Sbsq g sP P s2Sasq g

s by deﬁnition ofQE,g,B. Thus minq2CPs2Sasqs¼o¼Ps2Sbsq g

s¼Ps2Sasq g

s and soais an element of the r.h.s. In Step

3.2.C, the ﬁrst intersection produces the measures in arg minq2CPs2Sbsqs, and the remaining intersections ensure member-ship in QE,g,B_. _Given _qg₂_QE;g;B

\arg minq2CPs2Sbsqs, the ﬁrst intersection in Step 3.2.D results in the set bJextðfa2uBjf¼g on Ec _and P

s2Sasqgs ¼ogÞ. The remaining intersections in Step 3.2.D result in the set J¼ext co bJ\ a2RjSj_j₈_q₂_C_; P s2Sas qsq g s ð ÞP0 h i ¼ext uJg B

. The fact that in Step3.2.D, it sufﬁces to apply a ﬁnite number of intersections (8q2C and not "q2C) follows from Proposition A.1. Next observe that since KE;g;B E ¼Q E;g;B E \ q2CEj P s2SðasbsÞqs¼0; 8a2uJgB ; KE¼ext KE;g;B E

. Again, by Proposition A.1, it is sufficient in Step 3.2.Eto compute a finite number of intersections (8a2Jand not8a2uJgB). Finally, by definition ofUDC\PFImaxand

Proposi-tion A.1,CE,g,Bis the intersection ofCEand the sets q2CEjPs2Sðasb2ÞqðsÞP0

for alla2J.

Lemma A.1 (HK, p. 288, Lemma 3). Forð%;E;g;BÞ 2 T; QE;g;B

\arg minq2CRðugÞdq–;. The following proposition is needed for the proof ofProposition 3.3.

(11)

Deﬁnition A.1.Let RE;g;B ¼QE;g;B \arg min q2C Z ðugÞdq; andRE;g;B

E be the set of Bayesian conditionals onEof measures inR

E,g,B_{(see the proof of}_{Proposition 3.1}_{for the deﬁnition of} QE,g,B_).

Notation A.2. GivenEandh2 A, deﬁne the set of acts comparable and indifferent tohto be Jh

¼ ff2 Ajfhandf¼honEc

g:

Proposition A.2. UDC\RAmax_{is the update rule in}

UBayes_{such that}5

CE;g;B¼ qE q2Cand Z ðufÞdqEP min p2RE;g;B E Z ðugÞdpfor allf2Jg ( ) : Moreover, if uðXÞ ¼R, CE;g;B¼co qEjq2arg min p2C R uh ð Þdpfor someh2Jg and;for thath;RðuhÞdqEP min

p2RE;g;B E R ðugÞdp 8 > < > : 9 > = > ;:

Deﬁnition A.2.An actf0_is_{convexly related}_{to an act}_f_on_E_if_$_x₂_X_,_k₂_{[0, 1] such that}_f0₌_kf_{+ (1}_k₎_x_or_f₌_kf0_{+ (1}_k₎_x_on_E_.

Proof of Proposition A.2. The proof for the ﬁrst characterization follows an argument similar to the one in the proof of Proposition 5 in HK (p. 291) with RE,g,B playing the role of KE,g,B. To prove the second characterization, let D0

E;g;B¼ qEjq2arg minp2CRðufÞdpfor somef2Jgand;for thatf; RðufÞdqEPminp2RE;g;B E R ðugÞdp n o . LetC0_E;g;B¼coðD 0 E;g;BÞ. For any r2RE;g;B_;_rE₂_D0

E;g;B. Consider qE2D0E;g;B and associated act f2J

g _and _q₂_{arg min}

p2CRðufÞdp such that R

ðufÞdqEPminp2RE;g;B_E R

ðugÞdp. For suchqandfand anyh2Jg_, Z ðuhÞdqPmin p2C Z ðuhÞdp¼min p2C Z ðufÞdp¼ Z ðufÞdq; thusRðuhÞdqEP R ðufÞdqEPminp2RE;g;B_E R

ðugÞdp. Since these inequalities are preserved under convex combinations and closure,C0 E;g;B#CE;g;B and minp2C0 E;g;B R ðugÞdp¼minp2CE;g;B R

ðugÞdp. Moreover, we show that CE;g;B#C0E;g;B. Suppose that C0_E;g;B–CE;g;B. Since both sets are convex, there exists^q2CE;g;BnC0E;g;Bon the boundary ofCE,g,B. Thus there exists^f2Jgsuch thatq^2arg minp2CE;g;B

R

ðu^fÞdp. Since^qRC0

E;g;B, there exists such^ffor which R

ðu^fÞdq^<min_p₂_C0 E;g;B

R

ðu^fÞdp. If there exists h2Jg_{convexly related to}^_f_on_E_{such that min}

p2C0 E;g;B R ðuhÞdp¼minp2CE;g;B R ðugÞdpthenRðuhÞdq^<min_p₂_C0 E;g;B R ðuhÞdp¼ minp2CE;g;B R

ðugÞdp, contradicting ^q2CE;g;B. Thus for any h2Jg convexly related to ^f on E; minp2C0 E;g;B

R

ðuhÞdp>

minp2CE;g;B

R

ðugÞdp. Therefore for any such h and q₂_{arg min}

p2CRðuhÞdp; RðuhÞdqE>minp2CE;g;B R ðugÞdp. Thus q E2D 0

E;g;B. It follows that minp2C0 E;g;B

R

ðuhÞdp¼minp2CRðuhÞdpE by Lemma A.2. But then, minp2CRðuhÞdpE>

minp2CE;g;B

R

ðugÞdp, a contradiction. ThereforeC0

E;g;B¼CE;g;B, as required. h

Lemma A.2. Fix any f2Jg _{and let C}f

¼q2arg minp2CRðuhÞdpjh2Jg convexly related to f on E. If uðXÞ ¼R, then inf_p₂_CfRðufÞdp_E¼minp2CRðufÞdpE.

Proof.Ifufis constant onE, thenC¼arg minp2CRðufÞdpEand the lemma is trivially true. For the rest of the proof, we assumeufnon-constant onE. For such an actf2Jg_{, deﬁne the function}_b_:_ð₀_;_{1Þ !}_R_{at each}

a

_{> 0 by the solution to} minp2C

R

ð

a

ðufÞ þbð

a

ÞÞEðugÞdp¼minp2C R

ðugÞdp. Such a function is well-deﬁned becausep(E) > 0 for allp2C, and thus the left-hand side is strictly monotonic in b(

a

). Now deﬁne the function q : (0,1)?C such that, for

a

>0; qð

a

Þ 2arg minp2CRð

a

ðufÞ þbð

a

ÞÞEðugÞdp. We denoteq(

a

) byq

a_{. First we show that}R_ð_u_f_Þ_dqa

Eis non-increasing with

a

. Let

a

1>

a

2> 0. Then

5

Without the assumption uðXÞ ¼R, the set arg minp2CRðugÞdp should be replaced everywhere with its superset q2CjRðufÞdqP

R

ðugÞdqfor allf2Jg_g_{. As can be shown (proof available from the authors upon request), under the assumption}_u_ð_X_{Þ ¼}_R_{, the Bayesian conditionals formed} from the two sets are the same. Notice that the latter set depends onEwhile the former does not.

(12)

Z ð

a

1ðufÞ þbð

a

1ÞÞEðugÞdq a1_¼ Z ð

a

2ðufÞ þbð

a

2ÞÞEðugÞdq a2₆ Z ð

a

2ðufÞ þbð

a

2ÞÞEðugÞdq a1_; so ð

a

1

a

2Þ R EðufÞdq a1₆_½_b_ð

a

2Þ bð

a

1Þqa1ðEÞ. Similarly, ð

a

2

a

1Þ R EðufÞdq a2₆_½_b_ð

a

1Þ bð

a

2Þqa2ðEÞ. Thus R ðufÞdqa2 E P bða2Þbða1Þ a1a2 P R ðufÞdqa1 E , establishing that R

ðufÞdqa_Eis non-increasing with

a

. By observing that it is bounded, it follows that infa>0 RðufÞdqaE

exists and equals lima!1 RðufÞdqaE

. By deﬁnition ofb(

a

) andqa, for

a

> 0,

0¼ 1

a

qa_ð_E_Þ min_p₂_C Z ð

a

ðufÞ þbð

a

ÞÞEðugÞdpmin p2C Z ðugÞdp ¼ Z ðufÞdqaEþ bð

a

Þ

a

þ 1

a

qa_ð_E_Þ Z EcðugÞdq a min p2C Z ðugÞdp :

From this, lima!1

R ðufÞdqa_E ¼lima!1 bðaaÞaq1aðEÞ R EcðugÞdqaminp2C R ðugÞdp n o . Since R_EcðugÞdqa is bounded, lima!1 aq1aðEÞ R EcðugÞdqaminp2CRðugÞdp n o

¼0, and therefore, lima!1

R

ðufÞdqa_E

¼lima!1fbðaaÞg. From

minp2CRð

a

ðufÞ þbð

a

ÞÞEðugÞdp¼minp2CRðugÞdp, subtractingb(

a

) from each state and dividing by

a

gives min p2C Z ðufÞE ðugÞ bð

a

Þ

a

dp¼1

a

minp2C Z ðugÞdpbð

a

Þ : ðA:1Þ

Taking the limit as

a

?1on both sides of(A.1), min p2C Z ðufÞE _a!1lim bð

a

Þ

a

dp¼lim a!1 bð

a

Þ

a

; so lim a!1 bð

a

Þ

a

¼min p2C Z ðufÞdpE: Hence infa>0 R ðufÞdqa_E ¼minp2C R

ðufÞdp_E. IfuðXÞ ¼R, then actshasuch thatuha= (

a

(uf) +b(

a

))E(ug) exist for all

a

> 0 and each suchhais convexly related tofonEandha2Jg_{. Therefore, in this case, inf}

p2Cf R ðufÞdpE¼infa>0 R ðufÞdqaE

and the lemma is proved. h

Proof of Proposition 3.3. Ifq(E) = 1 for allq2C, then Step3.3.DproducescoðCEÞ ¼coðCÞ, which by inspection ofDCandRA, are the updated beliefs produced byUDC\RAmax_{in this case. From here on, suppose}_q₍_E_{) < 1 for some}_q₂_C_{. Let}_xEc

2X be a constant act for whichuðxEc

Þ ¼R_EcðugÞdqEc. We need to prove thatCE_,_g_,_Bhas as members the Bayesian updatesqEof all

q2Csatisfying (1)q(E)6_qg₍_E_{) if}_g_xEc_{, (2)}_q₍_E₎_P_qg₍_E_{) if}_xEc_g_{, or (3) no further restrictions if}_g_xEc_{. We use the second} description ofCE,g,Bpresented inProposition A.2. We ﬁrst show that measuresqthat violate (1)–(3) are not updated. Letf2Jg and let q2arg minp2CRðufÞdp. Also let b3minp2RE;g;B_E

R

ðugÞdp, i.e. b3¼

R

ðugÞdqg_E. Observe that since qg₂_RE;g;B_; _qg₂_{arg min}

p2CRðugÞdp. Also note thatxE

c

%_g_{implies that}_b

36uðxE

c

Þ, which is immediate ifqg₍_E_{) = 1, and} fol-lows also when qg₍_E_{) < 1 because 0}_PR_ð_u_g_Þ_dqg

R_EcðugÞdqg_Ec¼R_EðugÞdqgq g_ð_E_Þ qg_ð_Ec_Þ R EcðugÞdqg¼qgðEÞR_EðugÞdqg_E uðxEc ÞÞ. Addingf2Jg_{, we have} 0¼min p2C Z ðufÞdpmin p2C Z ðugÞdp¼ Z E ðufÞdqþ Z EcðugÞdq Z E ðugÞdqgþ Z EcðugÞdq g ¼ qðEÞ Z E ðufÞdqEuðxEc Þ þuðxEc Þ qg_ð_E_Þ b3uðxE c Þ þuðxEc Þ h i ¼qðEÞ Z E ðufÞdqEuðx Ec Þ qg ðEÞ b3uðx Ec Þ by hypothesis. So qðEÞR_EðufÞdq_EuðxEc Þ ¼qg_ð_E_Þ_b 3uðxE c Þ 60. Thus, if gxEc, then R EðufÞdqE¼uðxE c Þ ¼b3¼ min_p₂_RE;g;B E R ðugÞdp. IfxEc

g, all inequalities are strict and thusR_EðufÞdqEPminp2RE;g;B E

R

ðugÞdp()qðEÞPqg_ð_E_Þ_{. A} sim-ilar argument implies that ifgxEc

; R_EðufÞdq_EPmin_p₂_RE;g;B E

R

ðugÞdp()qðEÞ6qg_ðE_Þ. We now show that measuresqthat satisfy (1)-(3) are updated. First note that for anyf2 Asuch thatf=gonEc_and_q

2C, Z ðufÞdq¼ Z E ðufÞdqþqðEcÞuðxEc Þ ¼ Z ufExEc dq: ðA:2Þ If gxEc , then by taking f¼xEc

Eg and using equality (A.2) above we get fg and so f2J

g_{, and moreover}

8q2C; q2arg minp2CRðufÞdp. So "q2C,qE2CE,g,B, thus establishing (3). To prove (2), Suppose thatxE

c

(13)

for anyq2C, if there exist~q2C; k>1: ~qðFÞ ¼kqðFÞ; 8F#E, thenqand~qhave the same update, so it is sufﬁcient to con-siderqsuch that 9=~q2C; k>1: ~qðFÞ ¼kqðFÞ; 8F#E. ByLemma A.3which is proven below, for suchq, ifq(E)Pqg(E), then there exists f2 A constant on Ec_, _such _that _q

2arg minq02coðC[DðEc_ÞÞRðufÞdq0. So RðufÞdq6u_ðf_ðEc_ÞÞ since D(Ec₎_#_co₍_C

[D(Ec_)). _By _closure _of _CE

,g,B it is sufﬁcient to consider the case of strict inequality, so 0>RððufÞ uðfðEcÞÞÞdq¼bðminp2C

R

ðugÞdpuðxEc

ÞÞ for some b> 0. Let fExEc

2 A be such that ufExE c ¼1 bððufÞ uðfðE c ÞÞÞ þuðxEc

Þ. Thus q2arg minq0₂_CRu f_ExE c dq0 _and R_u fExE c dq¼minp2C R ðugÞdp, so fExEc

g, which by equality (A.2) implies that fEgg. Denoting f0₌_fEg_, _we _get _f0₂_Jg_. _Then R

Eðuf0ÞdqEPminp2RE;g;B E

R

ðugÞdpbyq(E)Pqg₍_E_{) and the property initially proved, so}_qE₂_CE

,g,B. The proof of case (1), wheregxEc

, is similar with the change (inLemma A.3and in the rest of the proof) thatqsatisﬁes q(E)6_qg₍_E_{) and}₉₌~_q₂_C_;_k_{2 ½}₀_;₁_Þ_{: ~}_q_ð_F_{Þ ¼}_kq_ð_F_Þ_;8_F_#_E_{, and}_co₍_C_[_D(_Ec_{)) is replaced by}_C0_{_kq00_{+ (1}_k₎

g

_:_q00₂_C_,

g

₂_D(_Ec_),

kP1}. Thus9f0_{2 A}_{constant on}_Ec_{such that}_q

2arg minq02C0Rðuf0Þdq0, soRðufÞdq>uðfðEcÞÞsincef0 separatesC0 from D(Ec_{) and then 0}_<R_ðð_u_f_Þ_u_ð_f_ð_Ec

ÞÞÞdq¼bðminp2CRðugÞdpuðxE

c

ÞÞfor someb> 0. h

Lemma A.3. Let q2C such that9=~q2C; k>1: ~qðFÞ ¼kqðFÞ;8F#E. Then there exists an act f 2 Awhich is constant on Ec_{, such} that q 2arg minq0₂coðC[DðEc

ÞÞ

R

ðufÞdq0.

Proof of Lemma A.3. If q(E) = 1, then any f2 A such that uf= 0E1 sufﬁces, because R

ðufÞdq0P0 for all q0₂_co₍_C_[_D(_Ec_{)) and} R

ðufÞdq¼0. From here on, suppose q(E) < 1. Consider the setW(q){q0 _: _q0₍_F_{) =}_q₍_F_), _"_F_#_E_} (note that C\W(q)–;). First assume Int(co(C[D(Ec)))\W(q)–;, i.e. there exists an interior point of co(C[D(Ec)),

q00_{, which satisﬁes} _q00₍_F_{) =}_q₍_F_), _"_F_#_E_{. Then} ₉

_>₀_{: ~}_q_ð₁_þ

_Þ_q00

₀_;_{. . .}_;₀

|fflfflfflffl{zfflfflfflffl} E ; 1 Ec ;. . .; 1 Ec |fflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflffl} Ec 0 B B B @ 1 C C C A2coðC[DðE c_ÞÞ_{. Setting}

k= 1 +

> 1, we get ~qðFÞ ¼kqðFÞ; 8F#E, which contradicts the initial condition. So Int(co(C[D(Ec₎₎₎_\_W(_q_{) =}_;_. Thus there exists a hyperplaneH which contains W(q) and supports co(C[D(Ec)) at any point in co(C[D(Ec))\W(q). Let us now take a normal uf to H. Since H contains W(q), it must be true that RðufÞdðqq0_{Þ ¼}_{0 for all}

q0₂_W(_q_{). So} 0¼ Z ðufÞdq Z ðufÞdq0¼ Z EcðufÞdq Z EcðufÞdq 0 ;

sinceqandq0_{are identical on}_E_{. Thus}R

EcðufÞdq0is constant regardless of the choice ofq0. Lets2Ec, and letq0_sbe such that

q0 sðFÞ ¼ qðFÞ; F#E qðEc Þ; s2F#Ec , soq0 s2WðqÞ; 8s2E c_{. Thus}₈_s_; _^_s 2Ec_{, we have}R EcðufÞdq0_s¼ R EcðufÞdq^0s ) ufðsÞ ¼ufð^sÞ, i.e.ufis constant onEc_{and therefore}_f_{can be chosen to be constant on}_Ec_. _h

Algorithm 4.1 (Step 3.3.D0_{). Fix some}_n₂_{(0, 1). Let}_A_{¼ ;}_{. For each pair}_q1_; _q2₂_C_{, solve the linear program,}

max a2RjEjK 0 X s2E ðasb3Þq2s; s:t: X s2S ðaEbÞsqsPo

8

q2C; X s2S ðaEbÞs nq1sþ ð1nÞq2s ¼o_; X s2E ðasb3Þq1s P0; ðP_Þ whereb3 P s2Ebs q g E sandK 0_{= 1 if}_K_P_{0 and}_K0₌_{1 if}_K_{< 0, for}_K_oP s2S½ðb31ÞEbs nq1sþ ð1nÞq2s . Consider only pairs (q1_,_q2_{) for which (}_P⁄_{) is feasible. If (}_P⁄_{) is unbounded, then add}_n_q1_{+ (1}

n)q2 _to_A_{. Otherwise, let} a⁄ 2argmax(P⁄_{). If} P s2E asb3 q2 s P0, then add q 1_, _q2 _to _A_{. Otherwise, add}

a

ðq1_;_q2_Þq1þ 1

a

_ð_q1_;_q2_Þ q2 _to _A_{, where}

a

ðq1;q2_Þ¼ P s2E a sb3 ð Þq2 s P s2E a sb3 ð Þq2 sq1s

ð Þ. ComputeUEextðfqEjq2AÞ. ReturncoðUEÞ.

Proposition A.3. If uðXÞ ¼R,Algorithm3.3with Step3.3.D0_{replacing Step}_3.3.D0_{results in the updated set of measures, CE,g,B,}

(14)

The following proposition is needed for the proof of Proposition A.3. In Proposition A.2, the condition q2arg minp2CPs2Saspsfor somea2uJg

will be referred to as condition 1. Forqwhich satisfy condition 1 with the vector a, the condition Ps2EasðqEÞsPb3minp2RE;g;B

E

P s2Ebsps

h i

will be referred to as condition 2. The following proposition eluci-dates the geometric properties of the updated beliefs.

Proposition A.4. If uðXÞ ¼R; CE;g;Bfor UDC\RAmaxequals the convex hull of Bayesian conditionals of

(a) Extreme points of C satisfying conditions 1 and 2.

(b)Non-extreme Boundary points of C which are convex combinations of two extreme points satisfying condition 1 with the same vector a, and for which condition 2 is satisﬁed with an equality.

Proof of Proposition A.4. Denote byDthe set of measures satisfying (a) or (b). LetCE

¼ fqjq2arg minp2CPs2Saspsfor some a2uJg_andP

s2Eðasb3ÞqsP0g. Since condition 2 is equivalent toPs2Eðasb3ÞqsP0; D#CE, thusco(D)#co(CE). We will show thatco(D) =co(CE). We ﬁrst show that an updated boundary point must be a proper (positive weight) convex com-bination of extreme points ofCsatisfying condition 1 with the samea. Any such boundary pointqcan be expressed as a proper convex combination of a setAof extreme points ofC, i.e.q¼Pi2Akiqi. Suppose thatqsatisﬁes condition 1 using the vector a and assume by contradiction that P_s₂_Sasqj

s>minp2CPs2Sasps¼ P

s2Sasqs for some j2A. Then P

s2Sasq0s< P

s2Sasqs forq011kj

P

i2Anfjgkiqi, contradicting q2arg minp2CPs2Sasps. Thus qi2arg minp2CPs2Sasps for alli2A. Next we show that for each edge (convex hull of two extreme points entirely on the boundary) ofC, the subset of updated points in this edge is convex. By the arguments above, it is sufficient to consider the case where both the edge’s vertices sat-isfy condition 1. LetRbe an edge ofC, such that one of its vertices,q1_{, satisfies conditions 1 and 2, and the other,}_q2_{, satisfies} only condition 1. If no point inRother thanq1_,_q2_{satisfies condition 1, then only}_q1_{gets updated, so this forms a convex set.} Now suppose that there exists a point in R other than q1_, _q2 _{satisfying condition 1. Let} _P₍_R_{) denote the set}

a2u ff2 Awithf¼gonEcgjq0₂

arg minp2CPs2Saspsfor allq0inRg. By the arguments above,P(R) is non-empty. For eacha2P(R), there exists a half-spaceP_s₂_Eðasb3ÞqsP0 corresponding to condition 2. This half-space does not contain the entire edgeR, for if it did, every point inRwould get updated, contrary to our assumption. So the intersection between the half-space andRmust be a set of the form

aq

1_{þ ð}₁

a

_Þ_q2_j

a

R6

a

61

(the weak inequality follows from closure of CE,g,B), where

a

aR>0 is the weight satisfying

P s2Eðasb3Þ

a

aRq1sþ 1

a

aR q2 s

¼0. Each vectora2P(R) has a unique corre-sponding weight

a

R. Thus the set of points inRwhich are updated is

aq

1þ ð1

a

Þq2j

a

R6

a

61

, where

a

Rinfa2PðRÞ

a

aR. This set is convex. Now consider a facetQ(convex hull of at least three extreme points entirely on the boundary) ofCsuch that all of its vertices satisfy condition 1, and some vertices, but not all, satisfy condition 2. LetD1(Q) denote the set of updated extreme points ofQ, and letD2(Q) denote the set of points of the form

a

Rq1þ 1

a

R

q2_in_Q_{. We show that the} set of points ofQwhich are updated is exactlyco(D2(Q)[D1(Q)), implying thatco(D) =co(CE). As we have seen above, it is impossible for a point to be updated, which is of the form

a

q1_{+ (1}

a

₎_q2_{, where}_q1_and_q2_{are the vertices of an edge}_R of Q, q2 _{does not satisfy condition 2, and}

a

_<

a

R, so assume int[Qnco(D2(Q)[D1(Q))] is non-empty. Consider a point

q2int[Qnco(D2(Q)[D1(Q))], and assume by contradiction thatqis updated. Then there exists a vectora2u{f2Bjf=g onEc} such thatq2arg minp2CPs2Sasps and

P

s2Eðasb3ÞqsP0. It follows from the same argument used earlier, that any q0₂_Q_{also belongs to arg min}

p2CPs2Sasps. Moreover, the half-space corresponding to condition 2 intersects any edgeRof Qinco(D2(Q)[D1(Q)), so the half-space intersects the whole facetQinco(D2(Q)[D1(Q)). But this contradicts the assumption onq, completing the proof. h

Proof of Proposition A.3. By construction of qg

E; b3¼minp2RE;g;B_E R

ðugÞdp. Denote P_s₂_Eðasb3Þq1s by h1(a) and

P

s2Eðasb3Þq2s byh2(a). The following is proved based onProposition A.4.

(1)Ifq1₌_q2_{, we show that}_(P⁄₎_{is bounded and moreover}_q1₌_q2_{is updated if and only if}_(P⁄₎_{is feasible with non-negative}

value. Since the second constraint in(P⁄₎_{implies that}_K₌_h

2(a),h2(a) is ﬁxed, so the program is bounded. Assume it is feasible. The ﬁrst and second constraints mean that there exists a vector aEb such that q1_¼_q2₂ arg minp2CPs2SðaEbÞsps. The second constraint means that minq2CPs2SðaEbÞsqs¼o, henceaEb2uJ

g

and condition 1 holds. The third constraint means that condition 2 holds forq1₌_q2_{, thus}_q1₌_q2_{is updated. On the other hand, if}

q1₌_q2_{is updated, it follows immediately from conditions 1 and 2 that}_(P⁄₎_{is feasible.}

(2)Ifq1–q2, there are several mutually exclusive and exhaustive cases:

a. Ifq1_,_q2_{fail to satisfy condition 1 with the same}_a_{(whether or not they are updated), we show that}_(P⁄₎_is

infea-sible. Assume by contradiction that(P⁄₎_{is feasible. Then verity of the ﬁrst and second constraints means that there}

exists a vector aEb2u ff2 A with f=g on Ec_{} such that} _nq1_{þ ð}₁_n_Þ_q2₂_{arg min}

p2CPs2SðaEbÞsps, and so due to arguments shown in the proof ofProposition A.4, bothq1_,_q2_{belong to arg min}

p2CPs2SðaEbÞsps. Verity of the second constraint means that minp2CPs2SðaEbÞsps¼o, soq

1

(15)

b. If onlyq2_{is updated and}_q1_{satisﬁes condition 1 with equal}_a_{but violates condition 2, then}_(P⁄₎_{is infeasible. This}

fol-lows again from contradiction of the third constraint.

c. If onlyq1_{is updated and}_q2_{satisﬁes condition 1 with equal}_a_{but violates condition 2, observe the following. First, by} the same arguments as above,(P⁄

)is feasible. We minimize

a

RforR=co{q

1

,q2} (see the proof ofProposition A.4). Since K=nh1(a) + (1n)h2(a), we haveh1ðaÞ ¼1nðK ð1nÞh2ðaÞÞ. Thus P s2Eðasb3Þð

a

aRq1sþ ð1

a

aRÞq2sÞ ¼0 implies

a

a R¼ P s2Eðasb3Þq2s P s2Eðasb3Þ q2s q1s ¼ h2ðaÞ h2ðaÞ h1ðaÞ ¼ h2ðaÞ h2ðaÞ K_nþ1_nnh2ðaÞ ¼ nh2ðaÞ h2ðaÞ K :

By the assumption onq1_and_q2_{, for all feasible}_a_,_h

1(a)P0 andh2(a) < 0. ConsequentlyKcould be of any sign, depend-ing on the choice ofn, so the behavior of the above expression as a function ofh2(a) depends on the sign ofK. IfKP0 (and soK0_{= 1), then the function is non-increasing, so minimization of}

a

Ris equivalent to maximization ofK0h2(a). If

K< 0 (thusK0₌

1