k)
2 = nσ and solving for n. Plugging this result into Theorem5.9yields an EVOI-loss bound for DMVM that applies to the special case of single-dimensional Gaussian uncertainty:
Lemma 5.10. Let the askable query set Q, and let ψ take the form of a Gaussian distribu-tion with varianceσ2. Then if DMVM selectsq,
maxq∗∈QEV OI(q∗; ψ) − EV OI(q; ψ) ≤
Note that since erf(x) is monotonically increasing in x, Equation5.5implies that pΩ(M (Q, D∗k)) is monotonically increasing in M (Q, D∗k) while monotonically decreasing in σ. This im-plies that for Gaussian distributions, the error bound stated in Lemma5.10prescribes lower worst-case EVOI-loss as 1) D∗kand Q become more related in terms of maximum mistake volume; and 2) the variance of the agent’s uncertainty increases. Intuitively, this can be understood as follows: the more specific the region of Ω the agent needs to learn about, the stronger the required connection between Dk∗ and Q (in terms of maximum mistake volume) in order to achieve the same upper bound on EVOI loss.
5.6 Summary of Algorithms and Results
Below I list the algorithms studied in this chapter and the main results I developed for them.
• Exhaustive (baseline)
– Computational complexity: O(|Q|kB + |Q|kΠ∗) – EVOI-loss: 0
• MEDER (baseline)
– Computational complexity: O |Q|kB + |Q|kE – EVOI-loss: ≤
1 − e−H(D∗|U |,Q)
Vmax− Vmin
• DEER (novel algorithm)
– Computational complexity: O(k(|U |kB + |Q|E)) – EVOI-loss: ≤
1 − e−H(D∗k,Q)
Vmax− Vmin
• DEER with first step approximated by greedy construction (novel algorithm)2 – Computational complexity: O(k(|U |Bk + k|Q|E))
– EVOI-loss: ≤
1 − e−H(D∗k,Q)
Vmax− Vmin
+(Vmax−Ve min)
• DMVM (novel algorithm)
– Computational complexity: setting-dependent – EVOI-loss: ≤
1 − e− log(k)pΩ M (Q,Dk∗);ψ
−1
Vmax− Vmin
5.7 Discussion
In this chapter I focused on developing principles for designing query selection algorithms that apply to any setting where the agent can select a query from some specified set of k-response queries. To this end, I introduced the Wishful Query Projection (WQP) approach for query selection, which operates by first computing the ideal query to ask assuming it can have only k possible responses, and then efficiently finding a similar askable query.
Intuitively, the main computational advantage of using WQP to select a query is that the query set can be expanded without requiring additional optimal planning computations, since all computations related to optimal planning are frontloaded into the first step. I then presented two specific implementations of WQP, DEER and an approximation of DEER called DMVM, and proved that both offer formal performance guarantees regarding the maximum loss in EVOI of the queries they select compared to the best one in the askable set, as a function of measures of how similar the askable query set is to the k-response decision query set. I also briefly discussed the impact that greedily approximating the first steps of DEER and DMVM has on their computational complexity, and also the impact that it has on their performance guarantees.
I examine the strengths and weaknesses of WQP (mainly focusing on DEER) further in Chapter 6, where I conduct an empirical study of its application (along with DMVM and MEDER) in a particular askable query selection setting. Before moving on to that more specific setting, however, it is useful to better understand how WQP approximates askable query selection in the abstract. Intuitively, WQP makes two approximations to
2Results apply only if recursive query improvement until convergence is performed after greedy construc-tion in the first step.
approximate EVOI-optimal query selection. The first approximation is that the askable query set is assumed to contain a query similar to the best k-response decision query d∗ computed in the first step. When this is not true, there is no guarantee that approximating d∗ will lead to a good query. For example, it could be the case that no askable query similar to d∗ exists, but that an askable query similar to another k-response decision query d exists, where although d and d∗ are dissimilar, d has nearly as high of value as d∗. The second approximation is that a query with higher similarity to d∗ is assumed to lead to higher EVOI, which may not be the case. For example, when using DEER, it may be that a specific set of models needs to be eliminated in order for the decision corresponding to one of the responses to d∗ to ever be valuable in expectation due to differences in relative value magnitudes among models, where at the same time the posterior response-entropy of d∗ is minimized by asking a query that eliminates a different set of models where the response to d∗ is equally uncertain even though the differences in value between decisions is unimpactful, resulting in the corresponding query having low EVOI. In the next chapter I will empirically study how these two approximations can affect DEER’s performance.
CHAPTER 6
Empirical Study of Wishful Query Projection in Askable k-Response Decision Query Selection
In Chapter4, I showed that when the agent has the ability to ask any query from the set of all k-response queries, the agent can consider only k-response decision queries at no EVOI penalty (Theorem4.2), which reduces the general k-response query selection problem to a tractable one where efficient, provably approximate algorithms apply. Then, in Chapter5, I restricted the agent by assuming that the agent can ask only queries lying within an arbitrary given subset of all k-response queries, where the agent cannot choose the subset. There, I showed that the EVOI-Sufficiency and tractability of the k-response decision query set can be leveraged to select from arbitrary k-response query sets in the form of two Wishful Query Projection (WQP) algorithms – DEER and DMVM. While I showed that DEER and DMVM have appealing computational properties while offering approximation guarantees, I have yet to provide empirical analysis of when using them would be suitable compared to alternative approaches.
In this chapter, I study the efficacy of WQP algorithms when applied to an extension of the standard decision query selection setting where the algorithms of Viappiani and Boutilier(2010) do not directly apply. Namely, I compare WQP algorithms to alternative approaches, such as an algorithm that attempts to make use of structure specific to this setting. To do so, I make liberal use of the algorithms, results, and terminology pertaining to the work ofViappiani and Boutilier(2010) presented in Section4.5of Chapter4.