Decision making under uncertainty using
imprecise probabilities
Matthias C.M. Troffaes
*Carnegie Mellon University, Department of Philosophy, 5000 Forbes Avenue, Baker Hall 135, Pittsburgh, PA 15213, USA
Received 3 February 2006; received in revised form 1 June 2006; accepted 2 June 2006 Available online 5 July 2006
Abstract
Various ways for decision making with imprecise probabilities—admissibility, maximal expected utility, maximality, E-admissibility,C-maximax,C-maximin, all of which are well known from the literature—are discussed and compared. We generalise a well-known sufficient condition for exis-tence of optimal decisions. A simple numerical example shows how these criteria can work in prac-tice, and demonstrates their differences. Finally, we suggest an efficient approach to calculate optimal decisions under these decision criteria.
Ó2006 Elsevier Inc. All rights reserved.
Keywords: Decision; Optimality; Uncertainty; Probability; Maximality; E-admissibility; Maximin; Lower prevision
1. Introduction
Often, we find ourselves in a situation where we have to make some decisiond, which we may freely choose from a setDof available decisions. Usually, we do not choosed arbi-trarily inD: indeed, we wish to make a decision that performs best according to some cri-terion, i.e., anoptimal decision. It is commonly assumed that each decision d induces a real-valued gainJd: in that case, a decisiondis considered optimal inDif it induces the
highest gain among all decisions inD. This holds for instance if each decision induces a
0888-613X/$ - see front matter Ó2006 Elsevier Inc. All rights reserved. doi:10.1016/j.ijar.2006.06.001
* Tel.: +1 412 268 3375; fax: +1 412 268 1440.
E-mail address:matthias.troff[email protected]
45 (2007) 17–29
lottery over some set of rewards, and these lotteries form an ordered set satisfying the axi-oms of von Neumann and Morgenstern[1], or more generally, the axioms of for instance Herstein and Milnor[2], if we wish to account for unbounded gain.
So, we wish to identify the set opt(D) of all decisions that induce the highest gain. Since, at this stage, there is no uncertainty regarding the gainsJd,d2D, the solution is simply
optðDÞ ¼arg max
d2D Jd: ð1Þ
Of course, opt(D) may be empty; however, if the set {Jd:d2D} is a compact subset ofR—
this holds for instance ifDis finite—then opt(D) contains at least one element. Secondly, note that even if opt(D) contains more than one decision, all decisionsdin opt(D) induce the same gainJd; so, if, in the end, the gain is all that matters, it suffices to identify only
one decisiond*in opt(D)—often, this greatly simplifies the analysis.
However, in many situations, the gainsJdinduced by decisionsdinDare influenced by
variables whose values are uncertain. Assuming that these variables can be modelled through a random variableXthat takes values in some setX(thepossibility space), it is customary to consider the gainJdas a so-called gamble on X, that is, we view Jd as a
real-valued gain that is a bounded function ofX, and that is expressed in a fixed state-inde-pendent utility scale. So,Jdis a boundedX–Rmapping, interpreted as an uncertain gain:
taking decisiond, we receive an amount of utilityJd(x) whenxturns out to be the
reali-sation ofX. For the sake of simplicity, we shall assume that the outcome xofXis inde-pendent of the decision d we take: this is called act-state independence. What decision should we take?
Irrespective of our beliefs aboutX, a decisiondinDis not optimal if its gain gambleJd
ispoint-wise dominatedby a gain gambleJefor someeinD, i.e., if there is aneinDsuch
thatJe(x)PJd(x) for allx2XandJe(x) >Jd(x) for at least one x2X: choosinge
guar-antees a higher gain than choosingd, possibly strictly higher, regardless of the realisation ofX. So, as a first selection, let us remove all decisions fromDwhose gain gambles are point-wise dominated (see[3, Section 1.3.2, Definition 5 ff., p. 10]):
optPðDÞ:¼ fd 2D:ð8e2DÞðJejJd orJe ¼JdÞg; ð2Þ whereJePJdis understood to be point-wise, andJejJdis understood to be the negation
of JePJd. The decisions in optP(D) are called admissible, the other decisions in D are
calledinadmissible. Note that we already recover Eq.(1)if there is no uncertainty regard-ing the gainsJd, i.e., if allJdare constant functions ofX. When do admissible decisions
exist? The set optP(D) is non-empty if {Jd:d2D} is a non-empty and weakly compact
subset of the setLðXÞof all gambles onX(seeTheorem 3further on). Note that this con-dition is sufficient, but not necessary.
In what follows, we shall try to answer the following question: given additional infor-mation aboutX, how can we further reduce the set optP(D) of admissible decisions? The
paper is structured as follows. Section2 discusses the classical approach of maximising expected utility, and explains why it is not always a desirable criterion for selecting optimal decisions. Those problems are addressed in Section3, discussing alternative approaches to deal with uncertainty and optimality, all of which attempt to overcome the issues raised in Section 2, and all of which are known from the literature. Finally, Section 4 compares these alternative approaches, and explains how optimal decisions can be obtained in a computationally efficient way. A few technical results are deferred toAppendix A, where
we, among other things, generalise a well-known technical condition on the existence of optimal decisions.
2. Maximising expected utility?
In practice, beliefs aboutXare often modelled by a (possibly finitely additive) proba-bility measurelon a fieldFof subsets ofX, and one then arrives at a set of optimal deci-sions by maximising their expected utility with respect to l; see for instance Raiffa and Schlaifer[4, Section 1.1.4, p. 6], Levi[5, Section 4.8, p. 96, ll. 23–26], or Berger[3, Section 1.5.2, Paragraph I, p. 17]. Assuming that the fieldFis sufficiently large such that the gains Jdare measurable with respect toF—this means that everyJdis a uniform limit ofF
-sim-ple gambles—the expected utilityof the gain gamblesJdis given by: ElðJdÞ:¼
Z
Jddl;
where we take for instance the Dunford integral on the right hand side; see Dunford[6, p. 443, Section 3], and Dunford and Schwartz [7, Part I, Chapter III, Definition 2.17, p. 112]—this linear integral extends the usual textbook integral (see for instance[8, Chapter 1]) to case wherelis notr-additive. Recall that we have assumed act-state independence: lis independent of d.
As far as it makes sense to rank decisions according to the expected utility of their gain gambles, we shouldmaximise expected utility:
optElðDÞ:¼arg max d2optPðDÞ
ElðJdÞ: ð3Þ
When do optimal solutions exist? The set optElðDÞis guaranteed to be non-empty if {Jd:
d2D} is a non-empty and compact subset of the setLðXÞof all gambles onX, with re-spect to the supremum norm. Actually, this technical condition is sufficient for existence with regard to all of the optimality conditions we shall discuss further on. Therefore, with-out further ado, we shall assume that {Jd:d2D} is non-empty and compact with respect
to the supremum norm. A slightly weaker condition is assumed inTheorem 5, inAppendix Aof this paper.
Unfortunately, it may happen that our beliefs aboutXcannot be modelled by a prob-ability measure, simply because we have insufficient information to identify the probprob-ability l(A) of every eventAinF. In such a situation, maximising expected utility usually fails to give an adequate representation of optimality.
For example, letXbe the unknown outcome of the tossing of a coin; say weonlyknow that the outcome will be either heads or tails (soX¼ fH;Tg), and that the probability of heads lays between 28% and 70%. Consider the decision set D= {1, 2, 3, 4, 5, 6} and the gain gambles J1ðHÞ ¼4; J1ðTÞ ¼0; J2ðHÞ ¼0; J2ðTÞ ¼4; J3ðHÞ ¼3; J3ðTÞ ¼2; J4ðHÞ ¼12; J4ðTÞ ¼3; J5ðHÞ ¼4720; J5ðTÞ ¼4720; J6ðHÞ ¼4110; J6ðTÞ ¼ 103:
Clearly, optP(D) = {1, 2, 3, 4, 5, 6}, and optElðDÞ ¼ f2g if lðHÞ<2 5; f2;3g if lðHÞ ¼2 5; f3g if 2 5<lðHÞ< 2 3; f1;3g if lðHÞ ¼2 3; f1g if lðHÞ>2 3: 8 > > > > > > < > > > > > > :
Concluding, if we havenoadditional information aboutX, but still insist on using a par-ticular (and necessarily arbitrary)l, which is only required to satisfy 0.286l(H)60.7, we find that optElðDÞis not very robust against changes inl. This shows that maximising ex-pected utility fails to give an adequate representation of optimality in case of ignorance about the precise value ofl.
3. Generalising to imprecise probabilities
Of course, if we have sufficient information such thatl can be identified, nothing is wrong with Eq. (3). We shall therefore try to generalise Eq. (3). In doing so, following Walley[9], we shall assume that our beliefs aboutXare modelled by a real-valued map-ping P defined on a—possibly only very small—set K of gambles, that represents our assessment of thelower expected utility P(f) for each gamble f in K;1note that K can be chosen empty if we are completely ignorant. Essentially, this means that instead of a single probability measure onF, we now identify a closed convex setMof finitely addi-tive probability measureslon F, described by the linear inequalities
ð8f 2KÞðPðfÞ6ElðfÞÞ: ð4Þ We choose the domainFof the measureslsufficiently large such that all gambles of inter-est, in particular those inK and the gain gamblesJd, are measurable with respect toF.
Without loss of generality, we can assumeFto be the power set ofX, although in prac-tice, it may be more convenient to choose a smaller field.
For a givenF-measurable gambleg, not necessarily inK, we may also derive a lower expected utilityEP(g) by minimisingEl(g) subject to the above constraints, and an upper expected utilityEPðgÞ ¼ EPðgÞby maximisingEl(g) over the above constraints. In case XandKare finite, this simply amounts to solving a linear program.
In the literature,M is called acredal set(see for instance[10], and[5, Section 4.2, pp. 76–78], for more comments on this model), andPis called alower prevision(because they generalise the previsions, which are fair prices, of De Finetti[11, Vol. I, Section 3.1, pp. 69–75]).
The mappingEP obtained, corresponds exactly to the so-callednatural extensionof P
(to the set of F-measurable gambles), where P(f) is interpreted as a supremum buying price forf(see[9, Section 3.4.1, p. 136]). In this interpretation, for anys<P(f), we are will-ing to pay any utilitys<P(f) prior to observation ofX, if we are guaranteed to receivef(x) oncex turns out to be the outcome ofX. The natural extension then corresponds to the
1
The upper expected utility of a gamblefisPðfÞif and only if the lower expected utility offisPðfÞ. So, for any gamblefinK,PðfÞ ¼ PðfÞ, and therefore, without loss of generality, we can restrict ourselves to lower expected utility.
highest price we can obtain for an arbitrary gamble g, taken into account the assessed pricesP(f) forf 2K. Specifically,
EPðgÞ ¼sup aþ Xn i¼1 kiPðfiÞ:aþ Xn i¼1 kifi6g ( ) ; ð5Þ
whereavaries over R,n overN,k1,. . .,knvary overRþ, andf1,. . .,fnoverK.
It may happen thatMis empty, in which caseEPis undefined (the supremum in Eq.(5) will always be +1). This occurs exactly whenP incurs a sure lossas a lower prevision, that is, if we can find a finite collection of gambles f1,. . .,fn in K such that Pni¼1PðfiÞ> sup Pni¼1fi
, which means that we are willing to pay more for this collection than we can ever gain from it, which makes no sense of course.
Finally, it may happen thatEPdoes not coincide withPonK. This points to a form of
incoherenceinP: this situation occurs exactly when we can find a finite collection of gam-blesf0,f1,. . .,fnand non-negative real numbersk1,. . .,kn, such that
aþX n i¼1 kifi6f0; but alsoPðf0Þ<aþ Xn i¼1 kiPðfiÞ:
This means that we can construct a price forf0, using the assessed pricesP(fi) forfi, which
is strictly higher thanP(f0). In this sense,EP correctsP, as is apparent from Eq.(5).
Although the belief model described above is not the most general we may think of, it is sufficiently general to model both expected utility and complete ignorance: these two extremes are obtained by takingM either equal to a singleton, or equal to the set of all finitely additive probability measures onF(i.e.,K¼ ;). It also allows us to demonstrate the differences between different ways to make decisions with imprecise probabilities on the example we presented before.
In that example, the given information can be modelled by, say, a lower prevision P on K¼ fIH;IHg, defined by P(IH) = 0.28 and P(IH) =0.7, where IH is the
gamble defined by IH(H) = 1 and IH(T) = 0. For this P, the set M corresponds exactly
to the set of all probability measures l on F¼ f;;fHg;fTg;fH;Tgg, such that 0.286l(H)60.7. We also easily find for any gamble fonXthat
EPðfÞ ¼minf0:28fðHÞ þ0:72fðTÞ;0:7fðHÞ þ0:3fðTÞg:
3.1.C-maximin andC-maximax
As a very simple way to generalise Eq.(3), we could take the lower expected utilityEP
as a replacement for the expected utilityEl(see for instance[12], or[3, Section 4.7.6, pp. 215–223]):
optEPðDÞ:¼arg max d2optPðDÞ
EPðJdÞ; ð6Þ
this criterion is calledC-maximin, and amounts to worst-case optimisation: we take a deci-sion that maximises the worst expected gain. For example, if we consider the decideci-sion as a game against nature, who is assumed to choose a distribution inMaimed at minimising our expected gain, then the C-maximin solution is the best we can do. Applied on the example of Section2, we find as a solution optEPðDÞ ¼ f5g.
In case K¼ ;, i.e., in case of complete ignorance about X, it holds that EPðfÞ ¼ infx2XfðxÞ. Hence, in that case, C-maximin coincides with maximin (see [3, Eq. (4.96), p. 216]), ranking decisions by the minimal (or infimum, to be more precise) value of their gain gambles.
Some authors consider best-case optimisation, taking a decision that maximises the best expected gain (see for instance [13]). In our example, the ‘‘C-maximax’’ solution is optE
PðDÞ ¼ f2g. 3.2. Maximality
Eq.(3)is essentially the result of pair-wise preferences based on expected utility: defin-ing the strict partial order >l on D as d>lewhenever El(Jd) >El(Je), or equivalently,
wheneverEl(JdJe) > 0, we can simply write
optElðDÞ ¼max>lðoptPðDÞÞ;
where the operator max>lðÞ selects the >l-maximal, i.e., the >l-undominated elements from a set with strict partial order >l.
Using the supremum buying price interpretation, it is easy to derive pair-wise prefer-ences fromP: define >Pasd>PewheneverEP(JdJe) > 0. Indeed,EP(JdJe) > 0 means
that we are disposed to pay a strictly positive price in order to take decisiondinstead ofe, which clearly indicates strict preference ofd overe(see[9, Sections 3.9.1–3.9.3, pp. 160– 162]). Since >Pis a strict partial order, we arrive at
opt>PðDÞ:¼max>PðoptPðDÞÞ ¼ fd2optPðDÞ:ð8e2optPðDÞÞðEPðJeJdÞ60Þg
ð7Þ
as another generalisation of Eq.(3), calledmaximality. Note that >Pcan also be viewed as
a robustification of >l over l in M. Applied on the example of Section 2, we find opt>PðDÞ ¼ f1;2;3;5g as a solution.
Note that Walley[9, Section 3.9.2, p. 161]has a slightly different definition: instead of working from the set of admissible decisions as in Eq.(7), Walley starts with rankingd>e ifEP(JdJe) > 0 or (JdPJeandJd5Je), and then selects those decisions fromDthat
are maximal with respect to this strict partial order. UsingTheorem 3fromAppendix A, it is easy to show that Walley’s definition of maximality coincides with the one given in Eq. (7)whenever the set {Jd:d2D} is weakly compact. This is something we usually assume to
ensure the existence of admissible elements; in particular, weak compactness is assumed in Theorem 5(seeAppendix A). The benefit of Eq.(7)over Walley’s definition is that Eq.(7) is easier to manage in the proofs inAppendix A.
3.3. Interval dominance
Another robustification of >l is the strict partial ordering AP defined by dAPe
wheneverEPðJdÞ>EPðJeÞ; this means that the interval½EPðJdÞ;EPðJdÞis completely on the right hand side of the interval½EPðJeÞ;EPðJeÞ. The above ordering is therefore called interval dominance(see[14, Section 2.3.3, pp. 68–69]for a brief discussion and references)
optAPðDÞ:¼maxAP optPðDÞ
¼ fd 2optPðDÞ:ð8e2optPðDÞÞðEPðJeÞ6EPðJdÞÞg:
The resulting notion is weaker than maximality: applied on the example of Section 2, optAPðDÞ ¼ f1;2;3;5;6g, which is strictly larger than opt>PðDÞ.
3.4. E-admissibility
In the example of Section 2, we have shown that optElðDÞ may not be very robust against changes inl. Robustifying optElðDÞagainst changes oflin M, we arrive at
optMðDÞ:¼ [
l2M
optElðDÞ; ð9Þ
this provides another way to generalise Eq.(3). The above criterion selects those admissi-ble decisions inDthat maximise expected utility with respect to at least onelinM; i.e., they select theE-admissible(see[15, p. 114, ll. 8–9], or[5, Section 4.8, p. 96, ll. 8–20]) deci-sions among the admissible ones. We find optMðDÞ ¼ f1;2;3gfor the example.
In case l is defined on }ðXÞ and l({x}) > 0 for all x2X, then every E-admissible decision is also admissible, and hence, in that case, optMðDÞgives us exactly the set of E-admissible options.
4. Which is the right one?
Evidently, it is hard to pinpoint the right choice. Instead, let us ask ourselves: what properties do we want our notion of optimality to satisfy? Let us summarise a few impor-tant guidelines.
Clearly, whatever notion of optimality, it seems reasonable to exclude inadmissible decisions. For ease of exposition, let us assume that the inadmissible decisions have already been removed fromD, i.e.,D= optP(D); this implies in particular that optMðDÞgives us
the set of E-admissible decisions.
Now note that, in general, the following implications hold:
as is also demonstrated by our example. A proof is given inAppendix A,Theorem 1. E-admissibility, maximality, and interval dominance have the nice property that the more determinate our beliefs (i.e., the smallerM), the smaller the set of optimal decisions. In contradistinction, C-maximin and C-maximax lack this property, and usually only select a single decision, even in case of complete ignorance. However, if we areonly inter-ested in the most pessimistic (or most optimistic) solution, disregarding other reasonable solutions, then C-maximin (orC-maximax) seems appropriate. Utkin and Augustin [16] have collected a number of nice algorithms for finding C-maximin and C-maximax solutions, and even mixtures of these two. Seidenfeld [17] has compared C-maximin to E-admissibility, and argued againstC-maximin in sequential decision problems.
If we do not settle forC-maximin (or C-maximax), should we choose E-admissibility, maximality, or interval dominance? As already mentioned, interval dominance is weaker than maximality, so in general we will end up with a larger (and arguably too large) set of optimal options. Assuming the non-admissible decisions have been weeded, a decisiondis not optimal inDwith respect to interval dominance if and only if
EPðJdÞ<sup e2D
EPðJeÞ: ð10Þ
Thus, ifDhas n elements, interval dominance requires us to calculate 2n natural exten-sions, and make 2n comparisons, whereas for maximality, by Eq.(7), we must calculate n2nnatural extensions, and performn2ncomparisons—roughly speaking, each nat-ural extension is a linear program inm(size ofX) variables andr(size ofK) constraints, or vice versa if we solve the dual program. So, comparing maximality and interval dom-inance, we face a tradeoff between computational speed and number of optimal options. However, this also means that interval dominance is a means to speed up the calcula-tion of maximal and E-admissible decisions: because every maximal decision is also inter-val dominant, we can invoke interinter-val dominance as a first computationally efficient step in eliminating non-optimal decisions, if we eventually opt for maximality or E-admissibility. Indeed, eliminating those decisionsdthat satisfy Eq.(10), we will also eliminate those deci-sions that are neither maximal, nor E-admissible.
Regarding sequential decision problems, we note that dynamic programming tech-niques cannot be used when using interval dominance (see [18]), and therefore, since dynamic programming yields an exponential speedup, maximality and E-admissibility are certainly preferred over interval dominance once dynamics enter the picture.
This leaves E-admissibility and maximality. They are quite similar: they coincide on all decision setsDthat contain two decisions. In case we consider larger decision sets, they coincide if the set of gain gambles is convex (for instance, if we considerrandomised deci-sions). As already mentioned, E-admissibility is stronger than maximality, and also has some other advantages over maximality. For instance,1
5J2þ 4
5J3>PJ5, so, choosing deci-sion 2 with probability 20% and decideci-sion 3 with probability 80% is preferred to decideci-sion 5. Therefore, we should perhaps not consider decision 5 as optimal.
E-admissibility is not vulnerable to such argument, since no E-admissible decision can be dominated by randomised decisions: if for somel2Mit holds that El(JdJe)P0
for alle2D, then also
El Jd Xn i¼1 kiJei ! ¼X n i¼1 kiElðJdJeiÞP0
for any convex combination Pni¼1kiJei of gain gambles, and hence, it also holds that
EP
Pn
i¼1kiJeiJd
60 which means that no convex combinationPn
i¼1kiJei can dominate Jdwith respect to >P.
A powerful algorithm for calculating E-admissible options has been recently suggested by Utkin and Augustin[16, pp. 356–357], and independently by Kikuti et al.[19, Section 3.4]. IfDhasnelements, finding all (pure) E-admissible options requires us to solven lin-ear programs inmvariables andr+nconstraints.
As we already noted, through convexification of the decision set, maximality and E-admissibility coincide. Utkin and Augustin’s algorithm can also cope with this case, but now one has to consider in the worst casen! linear programs, and usually several less:
the worst case only obtains if all options are E-admissible. For instance, if there are only‘ E-admissible pure options, one has to consider only at most‘! +n‘of those linear pro-grams, and again, usually less.
In conclusion, the decision criterion to settle for in a particular application, depends at least on the goals of the decision maker (what properties should optimality satisfy?), and possibly also on the size and structure of the problem if computational issues arise.
Acknowledgements
I especially want to thank Teddy Seidenfeld for the many instructive discussions about maximality versus E-admissibility. I also wish to thank two anonymous referees for their helpful comments. This paper has been supported by the Belgian American Educational Foundation. The scientific responsibility rests with its author.
Appendix A. Proofs
This appendix is dedicated to proving the connections between the various optimality criteria, and existence results mentioned throughout the paper. In the whole appendix, we assume the following.
Recall,Ddenotes some set of decisions, and every decisiond2Dinduces a gain gamble
Jd 2LðXÞ, whereLðXÞis the set of all gambles (boundedX–Rmappings).
Pdenotes a lower prevision, defined on a subsetKofLðXÞ. WithFwe denote a field onXsuch that all gain gamblesJdand gambles inKare measurable with respect toF,
i.e., are a uniform limit ofF-simple gambles.Fcould be for instance the power set ofX. Pis assumed to avoid sure loss, andEPis its natural extension to the set of allF
-mea-surable gambles.Mis the credal set representingP, as defined in Section3. We will make deliberate use of the properties of natural extension (for instance, superadditivity:
EP(f+g)PEP(f) +EP(g), and hence alsoEP(fg)6EP(f)EP(g)). We refer to Walley
[9, Section 2.6, p. 76, and Section 3.1.2, p. 123] for an overview and proof of these properties.
We use the symbollfor an arbitrary finitely additive probability measure onF, andEl denotes the Dunford integral with respect tol. This integral is defined on (at least) the set of allF-measurable gambles.
A.1. Connections between decision criteria
Theorem 1. The following relations hold: optE
PðDÞ optMðDÞ opt>PðDÞ optAPðDÞ;
optEPðDÞ opt>PðDÞ:
Proof. Let J¼ fJd: d 2Dg.
Suppose thatdisC-maximax inD:JdmaximisesEP in maxPðJÞ. SinceEP is the upper envelope ofM, andMis weak-*compact (see[9, Section 3.6]), there is alinMsuch that
EPðJdÞ ¼ElðJdÞ. But, ElðJeÞ6EPðJeÞ6EPðJdÞ ¼ElðJdÞ, for every Je2maxPðJÞ becaused isC-maximax. Thus,dbelongs to optMðDÞ.
Suppose thatd2optMðDÞ: there is a linM such thatJdmaximisesElin maxPðJÞ. But then, because EP is the lower envelope of M, EP(JeJd)6El(JeJd) =
El(Je)El(Jd)60 for allJein maxPðJÞ. Hence, by Eq.(7)on p. 7,dmust be maximal. Suppose that d is maximal. Then, again by Eq. (7), EP(JeJd)60 for all Je in maxPðJÞ. But,EPðJeÞ EPðJdÞ6EPðJeJdÞ, hence, alsoEPðJeÞ6EPðJdÞfor all Jein maxPðJÞ, which means thatdbelongs to optAPðDÞ.
Finally, suppose that d is C-maximin: Jd maximises EP in maxPðJÞ. But then EP(JeJd)6EP(Je)EP(Jd)60 for allJein maxPðJÞ;d must be maximal. h
A.2. Existence
We first prove a technical but very useful lemma about the existence of optimal ele-ments with respect to preorders; it’s an abstraction of a result proved by de Cooman and Troffaes[18]. Let us start with a few definitions.
Apreorderis simply a reflexive and transitive relation.
LetVbe any set, and let be any preorder onV. An elementvof a subsetSofVis called -maximalinSif, for allwinS,w vimpliesv w. The set of -maximal elements is denoted by
max ðSÞ:¼ fv2S:ð8w2SÞðw v)v wÞg: ðA:1Þ
For anyvin S, we also define theup-set ofvrelative toS as
"Sv:¼ fw2
S:w vg:
Lemma 2. LetVbe a Hausdorff topological space. Let be any preorder onVsuch that for any v in V, the set "Vvis closed. Then, for any non-empty compact subset S of V, the following statements hold:
(i)For every v in S, the set"Sv
is non-empty and compact. (ii) The set max ðSÞof -maximal elements ofSis non-empty.
(iii) For every v inS, there is a -maximal element w ofSsuch that w v. Proof
(i) Since is reflexive, it follows thatv v, so"Svis non-empty. Is it compact? Clearly,
"Sv¼
S\ "Vv, so "Svis the intersection of a compact set and a closed set, and
therefore"Svmust be compact too.
(ii) LetS0be any subset of the non-empty compact setSthat is linearly ordered with respect to . If we can show thatS0has an upper bound inSwith respect to , then we can infer from a version of Zorn’s lemma[20, (AC7), p. 144](which also holds for preorders) that Shas a -maximal element. Let then {v1,v2,. . .,vn} be an arbitrary finite subset ofS0. We can assume without loss of generality thatv1 v2 vn, and consequently "Sv
1 "Sv2 "Svn. This implies that the intersection Tn
k¼1" Sv
k¼ "Sv1of these up-sets is non-empty: the collectionf"Sv:v2S0gof com-pact and hence closed (Vis Hausdorff) subsets ofShas the finite intersection prop-erty. Consequently, sinceSis compact, the intersection Tv2S0"Svis non-empty as
well, and this is the set of upper bounds ofS0inSwith respect to . So, by Zorn’s lemma,Shas a -maximal element: max ðSÞis non-empty.
(iii) Combine (i) and (ii) to show that the non-empty compact set"Svhas a maximal
ele-mentwwith respect to . It is then a trivial step to prove thatwis also -maximal in S: we must show that for anyuin S, ifu w, thenw u. But, ifu w, then also u v since w v by construction. Hence, u2 "Sv, and since w is -maximal in
"Sv, it follows thatw u. h
The weak topology onLðXÞis simply the topology of point-wise convergence. That is, a netfainLðXÞconverges weakly tofin LðXÞif limafa(x) =f(x) for allx2X.
Theorem 3. IfJ¼ fJd: d2Dgis a non-empty and weakly compact set, then D contains at least one admissible decision, and even more, for every decision e in D, there is an admissible decision d in D such that JdPJe.
Proof. It is easy to derive from Eq. (2)that
optPðDÞ ¼ fd2D:ð8e2DÞðJe PJd)JdPJeÞg:
Hence, a decision is admissible inDexactly when its gain gamble isP-maximal inJ. We must show thatJhasP-maximal elements.
By Lemma 2, it suffices to prove that, for every f 2LðXÞ, the set Gf ¼ fg2LðXÞ:gPfg is closed with respect to the topology of point-wise convergence.
Letgabe a net inGf, and suppose thatgaconverges point-wise tog2LðXÞ: for every
x2X, limaga(x) =g(x). But, since ga(x)Pf(x) for every X, it must also hold that
g(x) = limaga(x)Pf(x). Hence, g2Gf. We have shown that every converging net inGf converges to a point in Gf. Thus,Gf is closed. This establishes the theorem. h
Let us now introduce a slightly stronger topology onLðXÞ. This topology has no par-ticular name in the literature, so let us just call it thes-topology. It is determined by the following convergence.
Definition 4. Say that a netfainLðXÞs-converges tofinLðXÞ, if (i) limafa(x) =f(x) for allx2X (point-wise convergence), and (ii) limaEPðjfafjÞ ¼0 (convergence inEPðj jÞ-norm).
This convergence induces a topologysonLðXÞ: it turnsLðXÞinto a locally convex topological vector space, which also happens to be Hausdorff. A topological basis at 0 consists for instance of the convex sets
ff 2LðXÞ:EPðjfjÞ< andfðxÞ<dðxÞg
for> 0, andd(x) > 0 for allx2X. It has more open sets and more closed sets than the weak topology, but it has less compact sets than the weak topology. On the other hand, this topology is weaker than the supremum norm topology, so it has fewer open and closed sets, and more compact sets, compared to the supremum norm topology. Note that
in caseXis finite, it reduces to the weak topology, which is in that case also equivalent to the supremum norm topology.
Note thatEP,EP, andElfor alll2M, ares-continuous, simply because
EPðjfafjÞPjEPðfaÞ EPðfÞj;
EPðjfafjÞPjEPðfaÞ EPðfÞj; and
EPðjfafjÞPjElðfaÞ ElðfÞj
(see[9, p. 77, Section 2.6.1(l)]). We will exploit this fact in the proof of the following the-orem, generalising a result due to Walley[9, p. 161, Section 3.9.2].
Theorem 5. IfJ¼ fJd : d 2Dgis non-empty and compact with respect to thes-topology, then the following statements hold:
(i) optElðDÞis non-empty for all l2M. (ii) optEPðDÞis non-empty.
(iii) optE
PðDÞis non-empty. (iv) opt>PðDÞis non-empty. (v) optAPðDÞis non-empty.
(vi) optMðDÞis non-empty.
Proof
(i) Introduce the following order on LðXÞ: say that f g wheneverEl(f)PEl(g). Let usz first show that, for all f 2LðXÞ, the set Gf ¼ fg2LðXÞ:g fg is s-closed.
Let ga be a net inGf, and suppose that gas-converges tog2LðXÞ. Since the integralEliss-continuous, it follows thatEl(g) = limaEl(ga)PEl(f). Conclud-ing,g belongs toGf. We have established that every converging net inGf con-verges to a point inGf. Thus, Gf iss-closed.
ByLemma 2, it follows thatJhas at least one -maximal elementJe, that is,Je maximisesEl in J. Since any s-compact set is also weakly compact, there is a P maximal element Jd in J such that JdPJe, by Theorem 3. But then,
El(Jd)PEl(Je), and hence,Jdalso maximisesElinJ. BecauseJdisP-maximal in J, it also maximises El in maxPðJÞ. This establishes that d belongs to
optElðDÞ: this set is non-empty.
(ii) Introduce the following order on LðXÞ: say that f gwhenever EP(f)PEP(g). Continue along the lines of (i), using the fact thatEPiss-continuous.
(iii) Again along the lines of (i), withf gwhenever EPðfÞPEPðgÞ. (iv)–(vi) Immediate, by (iii) andTheorem 1. h
References
[1] J. von Neumann, O. Morgenstern, Theory of Games and Economic Behavior, Princeton University Press, 1944.
[2] I.N. Herstein, J. Milnor, An axiomatic approach to measurable utility, Econometrica 21 (2) (1953) 291–297. [3] J.O. Berger, Statistical Decision Theory and Bayesian Analysis, second ed., Springer, 1985.
[4] H. Raiffa, R. Schlaifer, Applied Statistical Decision Theory, MIT Press, 1961.
[5] I. Levi, The Enterprise of Knowledge. An Essay on Knowledge, Credal Probability, and Chance, MIT Press, Cambridge, 1983.
[6] N. Dunford, Integration in general analysis, Transactions of the American Mathematical Society 37 (3) (1935) 441–453.
[7] N. Dunford, J.T. Schwartz, Linear Operators, John Wiley & Sons, New York, 1957.
[8] O. Kallenberg, Foundations of Modern Probability, second ed. Probability and Its Applications, Springer, 2002.
[9] P. Walley, Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London, 1991. [10] F.J. Giron, S. Rios, Quasi-Bayesian behaviour: a more realistic approach to decision making? in: J.M.
Bernardo, J.H. DeGroot, D.V. Lindley, A.F.M. Smith (Eds.), Bayesian Statistics, University Press, Valencia, 1980, pp. 17–38.
[11] B. De Finetti, Theory of Probability: A Critical Introductory Treatment, Wiley, New York, 1974–1975 (two volumes).
[12] I. Gilboa, D. Schmeidler, Maxmin expected utility with non-unique prior, Journal of Mathematical Economics 18 (2) (1989) 141–153.
[13] J.K. Satia, J. Roy, E. Lave, Markovian decision processes with uncertain transition probabilities, Operations Research 21 (3) (1973) 728–740.
[14] M. Zaffalon, K. Wesnes, O. Petrini, Reliable diagnoses of dementia by the naive credal classifier inferred from incomplete cognitive data, Artificial Intelligence in Medicine 29 (1–2) (2003) 61–79.
[15] I.J. Good, Rational decisions, Journal of the Royal Statistical Society, Series B 14 (1) (1952) 107–114. [16] L.V. Utkin, T. Augustin, Powerful algorithms for decision making under partial prior information and
general ambiguity attitudes, in: F.G. Cozman, R. Nau, T. Seidenfeld (Eds.), Proceedings of the Fourth International Symposium on Imprecise Probabilities and Their Applications, 2005, pp. 349–358.
[17] T. Seidenfeld, A contrast between two decision rules for use with (convex) sets of probabilities: Gamma-maximin versus E-admissibility, Synthese 140 (1–2) (2004) 69–88.
[18] G. de Cooman, M.C.M. Troffaes, Dynamic programming for deterministic discrete-time systems with uncertain gain, International Journal of Approximate Reasoning 39 (2–3) (2004) 257–278.
[19] D. Kikuti, F.G. Cozman, C.P. de Campos, Partially ordered preferences in decision trees: computing strategies with imprecision in probabilities, in: R. Brafman, U. Junker (Eds.), Multidisciplinary IJCAI-05 Workshop on Advances in Preference Handling, 2005, pp. 118–123.
