I now enumerate the main contributions I make in this dissertation, ordered by the chapters in which they appear.
1) Hybrid Algorithm for Action Query Selection in Sequential Decision-Making (Chap- ter3). The first contribution is the Hybrid algorithm, which is designed for action query selection in sequential decision-making settings. Hybrid utilizes the computational effi-
ciency of uncertainty reduction to select a promising subset of queries on which to per- form computationally expensive EVOI analysis. The efficacy of this Hybrid algorithm is demonstrated in the empirical investigations, where it displays clear advantages over pure uncertainty reduction and pure EVOI maximization in a setting considered in related work, supporting the hypothesis that hybrid uncertainty-based and EVOI-based techniques can be applied to feasibly perform EVOI-based query selection in problems of interest to the community.
2) EVOI-Sufficiency of k-Response Decision Query Set (Chapter 4). Another contri- bution is a theoretical result that narrows the space of queries that need even be consid- ered, provided maximizing EVOI is the explicit target and all other factors such as their human understandability/answerability are ignored. Namely, I show that given the only requirement is that queries must be constrained to having k possible responses, the set of k-response decision queries is sufficiently general in that there is no benefit in consider- ing any additional k-response queries. Practitioners can use this contribution to inform their design of query sets by restricting attention to decision queries, where efficient EVOI- based query selection algorithms exploiting submodularity have been recently developed. Of course, this contribution is directly useful only when decision queries can be sensibly asked and easily answered by humans in the target setting; however, even in such cases, the community can design query sets in a principled manner by designing them to be as informative about decision queries as possible, which connects with the next (third) contri- bution. In addition, this contribution forms the foundation for the fourth contribution below.
3) Response-Entropy Bound for EVOI-loss in Decision Query Projection (Chapter5). The next contribution establishes a means to characterize the worst-case loss in EVOI when using some k-response query in place of a decision query. Intuitively, the more a k-response query reduces uncertainty (in expectation) in what the response to a locally optimal deci- sion query would be, the smaller the loss in using the k-response query as a substitute for that k-response decision query. Researchers can use this contribution to assess the loss in EVOI associated with using some k-response query set that is suited to their target setting according to factors independent of EVOI (such as costs in developing interfaces for accu- rately presenting queries to humans and cognitive burdens imposed on humans to answer them), compared to if they were to use the k-response decision query set. In addition, this contribution, combined with the previous contribution, provides theoretical justification for the next contribution.
4) DEER Algorithm for Query Selection (Chapters 5 and 6). A fourth contribution draws on the previous two contributions to implement the Wishful Query Projection (WQP) approach for query selection, which is hypothesized in this dissertation as a means for tractably finding queries with high EVOI in settings where repeated optimal planning com- putations would be computationally prohibitive, in the form of a concrete query selection algorithm called Directed Expected Entropy Reduction (DEER). DEER begins by selecting an EVOI-optimal k-response decision query, which, according to the second contribution above, is the ideal k-response query to ask purely in terms of EVOI. However, this ideal query cannot be asked directly unless it is included by the target query set, and so the DEER algorithm selects the query from the target query set serving as the best substitute accord- ing to the third contribution above, and hence approximates EVOI-based query selection by restricting EVOI evaluations to an efficiently searchable (decision) query space and then finding a suitable match in the askable query set.
Applying the second and third contributions described above, DEER is shown to be principled in that it is guaranteed to select a query with EVOI close to the best query in the set, as a function of two different measures of the query set’s similarity to the k- response decision query set. DEER is also empirically evaluated by comparing it with baseline algorithms and several other approximate query selection algorithms, profiling its computational costs and performance abilities to delineate the contexts in which it pro- vides an effective tradeoff between good query choices and computational costs, as a func- tion of key properties of the query set to be selected from and the decision problem the agent faces, confirming the hypothesis that WQP can be implemented to produce an ef- ficient and approximate query selection algorithm both from a theroetical and empirical perspective. DEER thus contributes to the community a new query selection algorithm with well-understood strengths and limitations that can be particularly effective when (1) the computational cost of updating the agent’s uncertainty in light of query responses is small compared to evaluating the EVOI of a query, (2) the agent’s query set is balanced in the extent to which it contains queries similar to the range of k-response decision queries; and (3) the agent’s query set is rich in the extent to which it contains a query with similar EVOI to that of the EVOI-optimal k-response decision query.