A Note on Markov Decision Processes
Robust Markov Decision Processes
... ·|s, a) : a ∈ A for different actions in the same state may be dependent in an s-rectangular uncertainty set. By definition, (s, a)-rectangularity implies s-rectangularity. (s, a)-rectangular uncertainty sets have been ...
48
Sufficient Markov Decision Processes.
... to note that HTDN is trained using binary labels, whereas the other models are trained using ordinal labels and then have their ordinal predictions converted to binary ...
121
Compositional Reasoning for Markov Decision Processes
... Intuitively, we view each process term as describing a wMDP. Formally we describe one overarching wMDP where the states are all terms P in the grammar (2) and the weighted actions P −→ α w ∆ are those which can be ...
16
One-Counter Markov Decision Processes
... and the integer, d, resulting from this random sample is added to n ′ , obtaining the new pot of money n ′ + d. If the pot of money hits 0 or goes below zero, then the gambler loses (goes bankrupt) and the game ends. ...
36
Scalable Verification of Markov Decision Processes
... Figure 3 shows the empirical cumulative distribution of schedulers generated by Algorithm 3 applied to the MDP of Fig. 2, using p 1 = 0.9, p 2 = 0.5, ϕ = X(ψ∧ XG 4 ¬ψ), ε = 0.01, δ = 0.01 and M = 300. The vertical red ...
13
Hedging Bets in Markov Decision Processes
... value. Note that our algorithms generalize to these extensions immediately and to keep the presentation simple, we describe solutions only for the model defined ...
20
Augmenting Markov Decision Processes with Advising
... responsiveness, this map is abstracted into a 400 × 400- tiles hexagonal grid (any tile containing at least one obsta- cle pixel is considered as an obstacle tile). This grid is then transformed into a canonical ...
8
Bounded-parameter Markov decision processes
... on Markov decision processes; in the following, we mention just a few of the closest such ...(we note appealing to linear programming at each iteration can be very ...
39
Compositional reasoning for weighted Markov decision processes
... k = 2 1 k = ∞ . Lemma 2.26 (Distillation of Divergence — Static Case). In a finite-state wMDP if there is a hyper-SP-derivation ∆ =⇒ τ pp ,w ∆ ′ , there exists subdistribution ∆ ′ ε such that ∆ =⇒ τ w 1 ( ∆ ′ + ∆ ′ ε ) , ...
43
Markov Decision Processes and the Modelling of Patient Flows
... panels. Note the significant improvement in the proportion of days that are considered to have Very High occupancy; it drops to one percent or below for each day between Tuesday and Saturday ...
20
Trading performance for stability in Markov decision processes
... time. Note that if we knew the constant z, we would even get that the approximation problem for a point ( u , v ) can be solved in polynomial time (assuming that the number of digits in z is polynomial in the size ...
27
Policy gradient in Lipschitz Markov Decision Processes
... Table 2 shows how the changes in the parameters influence the number of iterations required to learn a near-optimal value of the policy parameter. The variability of the results comes from the estimation of the gradient ...
29
More Risk-Sensitive Markov Decision Processes
... The next corollary can be shown by induction. It states that the value iteration not only simplifies in the case of an exponential utility, but also in the case of a power or logarithmic utility. Note that part b) ...
20
Quantum partially observable Markov decision processes
... A QOMDP is fully observable in the same sense that the belief state MDP for a POMDP is fully observable. Just as the agent in a POMDP always knows its belief state, the agent in a QOMDP always knows the current quantum ...
11
Structural Results for Constrained Markov Decision Processes
... gaps are summarized in Tables 2.2, 2.3, and 2.4 for the first, second, and third parameter sets, respectively. The minimums and maximums in these tables are taken with respect to the abandonment rate. Note that a ...
159
Essays on semiparametric estimation of Markov decision processes
... related Markov decision models vary in th e literature, throughout th is note we follow the notations used in BBL when ...the decision problem of a single agent, an extension allowing for ...
193
Quantum partially observable Markov decision processes
... A QOMDP is fully observable in the same sense that the belief state MDP for a POMDP is fully observable. Just as the agent in a POMDP always knows its belief state, the agent in a QOMDP always knows the current quantum ...
12
Constrained Risk-Averse Markov Decision Processes
... Figure 3: Results for the MDP example with total expectation (left), CVaR (middle), and EVaR (right) coherent risk measures. The goal is located at the yellow cell. Notice the 9 single cell obstacles used for robustness ...
9
Multi-Objective Markov Decision Processes for Data-Driven Decision Support
... (2013) note that, ...the decision-maker to revisit action selection at each decision point in light of new information, both about state and about their own preferences and priorities over different ...
28
Strategy iteration algorithms for games and Markov decision processes
... to note that the strategy decision at these three vertices is irrelevant, because the same value is obtained no matter what edge is chosen at these vertices, and the balance will always be ...
226