Random comparisons

The single scenario we presented in the previous section may not tell the whole story. Next, we will compare these policies with randomly generated parameters. We randomly generate 1000 sets of parameters, with all probabilities µi and βi uniformly in (0.1,0.5), pii ∼(1−µi)U(0.8,1) and qii ∼(1−βi)U(0.8,1). We

takes value in{10,12,14,16,18,20} for each set of the parameters. We generated a total of 6000 scenarios, and the following analysis will be conducted based on these 6000 scenarios.

We first look at how each static policy performs compared with the optimal policies. Let P1 denote the static policy that always prioritize stage 1 customers, and P2 denote the policy that always prioritize stage 2 customers. Let g1 and g2 denote the respective long-run average reward for policy P1 and P2, andg∗

denote the long-run average reward under the optimal policy. We conduct 95% confidence intervals (C.I.) of the relative differences (g∗−g1)/g∗, (g∗−g2)/g∗ and (g∗−max(g1, g2))/g∗, and the results are shown in

Table 4.4 (all numbers are in percentage).

Table 4.4: Comparisons of the static policies and the optimal policy

(g

−g

/g

(g

−g

/g

(g

−max(g

, g

/g

95% C.I.

(6.6360, 7.2617)

(7.1521, 7.7919)

(0.0048, 0.0074)

Maximum

69.8460

72.1264

1.2222

We find that the best static policy performs nearly optimal. However, given the customers change stages in both service and queue, it is difficult to analytically compute and compare the long-run average rewards of P1 and P2. Hence, we focus on whether we could find some criteria that is easy to employ and determine which static policy should be chosen. The priority index policies provides in Table 4.1 are some of such criteria, and we next compare the performances of these policies. As before, we first provide the 95% C.I. of the relative differences in long-run average reward under proposed policy and the optimal policy, the maximum of the relative difference, and we also compute the percentage of scenarios that the proposed policy is consistent with the best static policy. More specifically, we conduct C.I.s and find maximum of the quantity (g∗_−g

π)/g∗wheregπ denote the long-run average reward for policyπ. Let Nπdenote the number

of scenarios that policyπis consistent with the best static policy, and we computeN/6000 as the percentage of scenarios that the proposed policy is consistent with the best static policy. The results are provided in Table 4.5 (all numbers are in percentage).

Table 4.5: Comparisons of the proposed index policies

Policy

Index

95% C.I. of (g

−g

)/g

Maximum of (g

−g

)/g

N

/6000

R

R

(4.24, 4.73)

60.64

65.08

OSR

R

µ

(0.08, 0.10)

6.35

93.05

RR

R

/L

(0.21, 0.28)

28.05

90.98

RR/AR

R

L

/L

(2.49, 2.83)

55.89

69.97

RRAR

R

/(L

L

)

(1.22, 1.46)

48.10

84.58

EDRD

R¯

−R¯

(5.12, 5.69)

69.85

63.77

We found that in this study, the one-step-reward policy outperforms all the other index policies.

4.8 Conclusions

Customers are impatient and may change stages in many service systems. There are many parameters that could affect the prioritization decisions, such as service rates and rewards, abandonment rates, transition probabilities, etc. We aim to develop a model to see how the priority should be assigned under different settings of these parameters, and propose some priority index policies based on our model analysis.

We first formulate the system an MDP model. We find that our formulation does not guarantee the optimality of non-idling, which should be the case in reality for most service systems. We consider the case when there are ample servers and we find the necessary and sufficient condition under which non-idling should be optimal. Intuitive explanation for this condition is that delaying the service for one period of time will reduce the expected discounted reward obtained for every customer. We next consider the general case when server capacity is constrained. With an additional assumption on the transition probabilities (may not be necessary), we are able to provide a sufficient condition under which non-idling is optimal. Next, we compare the actions of serving a stage 1 customers versus the action of serving a stage 2 when a server is available, and find that the action of service a stage 1 is always better than serving a stage 2 when a server is available under certain conditions. Hence, these conditions are sufficient conditions under which it is optimal to always prioritize stage 1 customers.

We consider several special settings of the general model. The first setting is to assume no queueing for the system. Then, the model reduces toModel Ithat is very similar to the one in Chapter 3, with slightly different reward structure. However, utilizing the similar structure of the optimality equations, we conclude that the optimal policy forModel Iis actually the same as the ICU model in Chapter 3. The second special parameter setting considered inModel IIis the case when there is no transition between stages. We obtain

a sufficient condition under which always prioritizing one stage is optimal by applying our general model results. We also extend the result by considering reneging also happens for customers in service inModel II(A), and obtain a result that is consistent with that in Down et al. (2011), but ourModel II(A)is more general since we consider a multi-sever and non-identical service times. The third special parameter setting study the model in Chapter 5.2 of Jacobson (2010), and provide a similar result for a multi-server system.

We propose several index policies from our model analysis, and conduct a numerical study to compare the performances. In this study, we first find the switching curve structure of the optimal policy and observe how the switching curve changes with respect to different parameters. We conduct 6000 random parameter combinations and compare the performance of different index policies. We find that in our study, the priority index policy that prioritizes the stage with largest one-step expected reward performs the best among all index policies we proposed.