Priorities Modeled as Quasi-Birth-Death Processes
Amir Rastpour
Faculty of Business and Information Technology, Ontario Tech University [email protected]
Armann Ingolfsson
Alberta School of Business, University of Alberta, [email protected]
Burhaneddin Sandık¸cı
Booth School of Business, University of Chicago, [email protected]
We consider a Markovian multi-server queueing system with two customer classes, preemptive priorities, and reneging. We formulate this system as an infinite level-dependent quasi-birth-death process (LDQBD).
We introduce an algorithm that endogenously truncates the level and calculates lower and upper bounds on stationary probabilities of this LDQBD such that the gap between the bounds can be any desired amount.
Our algorithm can be applied to any LDQBD for which the rate matrices become element-wise non-increasing above some level. This appears to be the first algorithm that provides bounds on stationary probabilities for an infinite-level LDQBD. To obtain these bounds, the algorithm first obtains lower and upper bounds on the rate matrices of the LDQBD using a novel method, which can be applied to any LDQBD. We extend this algorithm to approximate performance measures of the system of interest, and calculate exact lower and upper bounds for those that can be expressed as probabilities, such as the probability that an incoming low-priority customer will wait. We generate a wide range of instances with up to 100 servers and compare the solution time and accuracy of our algorithm with two existing algorithms. These numerical experiments indicate that our algorithm is faster than the other two algorithms for a given accuracy requirement. We investigate the impact of changing service rates on the proportion of Class-2 customers served and their wait time, and demonstrate how ignoring one of these measures can possibly mislead decision makers.
Key words : Service management, Priority Erlang A, Level-dependent QBD, Bounding
1. Introduction
Priority queues with impatient customers provide a powerful framework for formulating and computing performance measures for a variety of real-world systems. However, due to their high level of complexity, performance evaluation algorithms have only been developed for special cases, such as single-server systems (Iravani and Balcıo˘glu 2008, Drekic et al. 2015) and systems with identical service and patience time distributions for all customer classes (Jouini and Dallery 2007, Jouini and Roubos 2014). We focus on generic s-server priority queues with two impatient customer classes having memoryless, but possibly different arrival,
1
service and patience processes. We adopt the priority queue notation from Takahashi and Hashida (1991) and use M/M/s + M (Pr) to denote our system of interest.
We model this system as a quasi-birth-death process (QBD), which is an extension of the birth-death process from a univariate state space to a bivariate state space, where the first coordinate is called the level and the second coordinate is called the phase. There are two main classes of QBDs, namely level-independent and level-dependent. QBDs with finite and infinite state spaces lend themselves to a wide range of application areas, including emergency departments (Campello et al. 2016), systems with load and overwork-dependent service rates (Delasay et al. 2016), and tumor recurrence (Santana et al. 2019).
Level-independent QBDs are analogous to birth-death processes with a geometric tail, leading to closed-form expressions for various performance measures. These closed-form expressions involve the so-called rate matrix, which has to be computed numerically. There is extensive research on properties and algorithms for level-independent QBDs; see Ciardo and Smirni (1999) and classical books by Neuts (1981) and Latouche and Ramaswami (1999).
Level-dependent QBDs (LDQBDs) are analogous to birth-death processes with rates that vary with the state variable, such as M/M/∞ and M/M/c + M (Erlang A) queues. Except for the M/M/∞ special case, closed-form expressions typically do not exist for performance measures of such birth-death systems; the same is true for LDQBDs. The level and phase coordinates of LDQBDs can be finite or infinite, and these models have been used in many application areas including organ donation (Drekic et al. 2015), chemical reactions (Dayar et al. 2011), epidemics (Baumann and Sandmann 2016), tandem queues with abandonment (Wang et al. 2019), and retrial queues (Latouche and Ramaswami 1999, Section 1.8). Our QBD model in this paper is level dependent and has an infinite state space, with infinite levels and finite phases. In the remainder of the paper, we use LDQBD to refer to such QBDs.
Numerical analysis of LDQBDs has received less attention than that for level-independent QBDs. As noted by Kharoufeh (2011), calculation of steady-state probabilities for general LDQBD involves two major complications: (i) the state space must be truncated in an appropriate manner, and (ii) the level-dependent rate matrices need to be computed effi- ciently. Existing solution methods (Takine 2016, Bright and Taylor 1995, Baumann and Sandmann 2013) have two main shortcomings: (i) they truncate the level exogenously (and somewhat arbitrarily), and (ii) no guarantees are offered for the resulting errors in steady- state probabilities. We address both of these shortcomings for a practically important class of LDQBDs.
We introduce an algorithm that endogenously truncates the level and calculates lower and upper bounds on stationary probabilities of LDQBDs associated with M/M/s + M (Pr) systems, with any desired error tolerance. This appears to be the first algorithm that provides such bounds on stationary probabilities of an LDQBD. To obtain these bounds, the algorithm first obtains lower and upper bounds on the rate matrices of the LDQBD using a novel method.
Our method to bound the rate matrices is applicable to any LDQBD, and our method to bound the stationary probabilities is applicable to any LDQBD whose rate matrices become element-wise non-increasing above some level. We generate a wide range of instances with parameters that push the limits of our algorithm in order to identify conditions under which it fails. Using these instances, we compare the solution time and accuracy of our algorithm with two existing algorithms. We find that Algorithm 1 (Baumann and Sandmann 2013) is faster, but inaccurate (with more than 10% relative error for more than half of the instances), and Algorithm A2 (Shin and Choo 2009) is highly accurate, but slower (14 times, on average), in comparison to our algorithm. We also demonstrate that when customers are impatient, evaluating the impact of changing the service rate of one customer class versus the other one, solely based on the waiting time of Class-2 customers without considering the proportion of those who do receive service, can be misleading.
The remainder of the paper is organized as follows. We provide background on QBDs and discuss how we model the M/M/s + M (Pr) system as an LDQBD in Section 2. We introduce our algorithm in Section 3, establish its correctness in Section 4, and extend it to calculate performance measures in Section 5. We provide numerical results in Section 6, provide insights into behaviour of an M/M/s + M (Pr) system in Section 7, and conclude in Section 8. All proofs are provided in the Appendix.
2. Models and Definitions
We introduce the notation and definitions in Section 2.1, model the M/M/s + M (Pr) system as an LDQBD in Section 2.2, and derive its stationary probabilities in Section 2.3.
2.1. Definitions
Consider a continuous-time Markov chain {Xt, t ∈ R+} on the state space S = {(`, p) : ` ∈ Z+, p ∈ H}, where Z+= {0, 1, ...} and H = {0, 1, ..., p∗}, for a given finite positive integer p∗. This stochastic process is a QBD if one-step transitions from state (0, p) are restricted to states (0, p0) and (1, p0), and transitions from state (`, p), for ` ∈ Z++= Z+\{0} are restricted
to states (` − 1, p0), (`, p0), and (` + 1, p0), for p, p0 ∈ H (Latouche and Ramaswami 1999, Chapter 6). That is, a one-step transition can move the state of the QBD to any phase within the current level, one level up, or one level down.
Let the indicator function 1A(x) be 1 when x satisfies condition A, and 0 otherwise; I an identity matrix, and 0 and 1 matrices of zeros and ones, respectively, of appropriate dimensions. We use uppercase letters to denote matrices, and lowercase letters to denote vectors and scalars. Inequalities are taken as element-wise when applied to vectors and matrices. For a matrix X = (xij), we use X and X to denote matrices of element-wise lower and upper bounds on X, respectively, xi. to denote the ith row sum, X> to denote transpose, kXkmax= maxij|xij| to denote the max norm, and σ(X) to denote the spectral radius (the largest absolute value of the eigenvalues). Table A1 lists our notation.
The infinitesimal generator matrix Q of an LDQBD is block tridiagonal:
Q =
`=0 `=1 `=2 `=3 ...
`=0 Q(0)↔ Q(0)↑
`=1 Q(1)↓ Q(1)↔ Q(1)↑
`=2 Q(2)↓ Q(2)↔ Q(2)↑
`=3 Q(3)↓ Q(3)↔ . ..
... . .. ...
, (1)
where matrix blocks Q(`)↑, ` ∈ Z+, contain transition rates from level ` to ` + 1 (↑ indicates that level increases); blocks Q(`)↔, ` ∈ Z+, contain within-level transition rates (↔ indicates that the level does not change); and blocks Q(`)↓, ` ∈ Z++, contain transition rates from level
` to ` − 1 (↓ indicates that level decreases). All blocks are (p∗+ 1) × (p∗+ 1) matrices. We use qp,p(`)i0 to denote the (p, p0)th element of Q(`)i, for i ∈ {↑, ↔, ↓}, ` ∈ Z+ and p, p0∈ H. We also use qp.(`)= −q(`)↔p,p to denote the total rate by which the system leaves state (`, p), which implies 1/qp.(`) is the average sojourn time for state (`, p). Level-independent QBDs restrict the blocks to be constant across all levels ` ≥ 2; that is, Q(`)i = Qi, for i ∈ {↑, ↔, ↓} and
` ∈ {2, 3, ...}, while blocks of levels ` ∈ {0, 1} may be different due to boundary conditions.
2.2. M/M/s + M (Pr) System as an LDQBD
In this subsection, we formulate an LDQBD model for the M/M/s + M (Pr) system and construct its infinitesimal generator matrix Q. This queueing system has s servers and two customer classes, each with an independent Poisson arrival process with rate λi, i.i.d.
exponential service time with rate µi, and i.i.d. exponential patience time with rate γi,
i = 1, 2. The vector θ = (λ1, λ2, µ1, µ2, γ1, γ2, s) fully specifies the system. Class-1 customers have preemptive-restart priority over Class-2 customers. That is, Class-2 customers can receive service only if there is no Class-1 customer in the system, and if a Class-2 customer is receiving service when a Class-1 customer arrives, then the Class-2 customer immediately returns to the queue, freeing the server for the arriving Class-1 customer.
To model this system as a QBD, we associate Class-2 customers with the level and Class-1 customers with the phase, and let ` and p denote the number of Class-2 and Class-1 customers in the system, respectively. This assignment simplifies the calculation of the phase truncation error, because the preemptive policy implies that the Class-1 sub-system evolves independently of Class-2 customers, and it would form an Erlang-A system if no limit was imposed on the count of Class-1 customers. We choose the phase limit p∗ such that the probability of having more than p∗ customers in an infinite-capacity Erlang A system is at most p. Based on the work of Ingolfsson and Tang (2012), we present Algorithm A1 to compute p∗.
Let s1= min{p, s} and s2= min{`, s − s1} be the number of Class-1 and Class-2 customers receiving service, respectively. Furthermore, let q1= p − s1 and q2= ` − s2 be the number of Class-1 and Class-2 customers in the queue, respectively. Possible transitions from state (`, p) are illustrated in Figure 1, and the elements of Q for p, p0∈ H and ` ∈ Z+ are presented as:
qp,p(`)↑0 =
λ2 if p0= p, 0 otherwise,
qp,p(`)↔0 =
λ1 if p0= p + 1 and p < p∗, s1µ1+ q1γ1 if p0= p − 1
−q(`)p. otherwise,
(2a)
qp,p(`)↓0 =
s2µ2+ q2γ2 if p0= p,
0 otherwise.
(2b)
The main computational challenge in analyzing LDQBDs is the calculation of the prob- abilities in the infinite tail (` → ∞) of the two-dimensional state space, and therefore we pay special attention to the block structure for large `. For ` ≥ s, all block elements remain constant, except the diagonal elements of Q(`)↓ increase by γ2, and the diagonal elements of Q(`)↔ decrease by γ2, for each increment in the level. That is, for ` ≥ s,
Q(`+1)↑= Q(`)↑= λ2I, (3a)
Q(`+1)↔= Q(`)↔− γ2I, (3b)
Q(`+1)↓= Q(`)↓+ γ2I. (3c)
Phasep:Class-1customers
Level `: Class-2 customers
maximum phase (p∗)
`, p ` + 1, p
`, p + 1
`− 1, p
`, p− 1 [λ2] qp,p(`)↑
q(`)↔p,p+1 [λ1· 1<p∗(p)]
q(`)↓p,p
[s2µ2+ q2γ2]
q(`)↔p,p−1 [s1µ1+ q1γ1]
Figure 1 Possible transitions from state (`, p) for an M/M/s + M (Pr) system.
We illustrate blocks (3a)-(3c), for ` ≥ s and p∗> s, in Appendix A.
2.3. LDQBD Stationary Probabilities The rate matrices R(`)=r(`)pp0
, ` ∈ Z+, p, p0∈ H, play an important role in analyzing LDQBDs.
The element r(`)pp0 is the average sojourn time in state (` + 1, p0) before the first return to level
` per unit time spent in state (`, p) provided that the system started at (`, p). The matrices R(`) can be computed recursively (Bright and Taylor 1995):
R(`)= Q(`)↑−Q(`+1)↔− R(`+1)Q(`+2)↓−1, ` ∈ Z+. (4) Let π = π(0), π(1), ... be the stationary distribution of the LDQBD, where π(`) =
π0(`), π(`)1 ..., πp(`)∗, and πp(`) is the stationary probability of state (`, p), for ` ∈ Z+ and p ∈ H.
Given rate matrices R(`), ` ∈ Z+, we can recursively compute the probability vector π(`) (Latouche and Ramaswami 1999):
π(`+1)= π(`)R(`), ` ∈ Z+. (5a)
The steady-state probability vector π is the solution to πQ = 0 and π1 = 1. The ` = 0 component of the πQ = 0 system is
π(0)Q(0)↔+ R(0)Q(1)↓= 0. (5b) Using (5a), the normalization condition π1 = 1 yields
∞
X
`=0
π(`)
!
1 = π(0)I + R(0)+ R(0)R(1)+ · · ·1 = 1. (5c)
Algorithm 1: Baumann and Sandmann (2013)
1 Input: truncation level `∗ and θ
2 initialization: ˆR(`∗)= 0
3 for ` = `∗− 1 to 0 do
4 compute ˆR(`)= Q(`)↑
−Q(`+1)↔− ˆR(`+1)Q(`+2)↓−1
// [eq. (4)]
5 end
6 solve ˆx(0)
Q(0)↔+ ˆR(0)Q(1)↓
= 0 to obtain ˆx(0) // [eq. (5b)]
7 for ` = 0 to `∗− 1 do
8 compute ˆx(`+1)= ˆx(`)Rˆ(`) // [eq. (5a)]
9 end
10 compute the normalizing constant: ˆc = P`∗
`=0xˆ(`)
1 // [eq. (5c)]
11 return: ˆπ(`)= ˆx(`)/ˆc, for ` = 0, ..., `∗.
Baumann and Sandmann (2013) developed Algorithm 1 to approximate the stationary distribution π. They start with truncating the system at an exogenously-chosen level `∗< ∞, and then recursively solve (4) to calculate ˆR(`) as an estimate of R(`). Solving (5b) with R(0)= ˆR(0), they calculate ˆx(0) as an unnormalized estimate of π(0), which is substituted in (5a) to recursively calculate unnormalized estimates ˆx(`) for π(`), for ` = 1, ..., `∗. Then, they divide ˆx(`) by an estimate of the normalizing constant, ˆc, to obtain the estimates ˆπ(`), for
` = 0, 1, ..., `∗. This procedure has two major limitations: it is not clear (i) how to choose a value of `∗ for the truncation level, and (ii) how truncation impacts the accuracy of the estimate ˆπ. In Section 3, we describe an algorithm that addresses these limitations for our LDQBD model of the M/M/s + M (Pr) system and, more generally, for any LDQBD whose rate matrices become element-wise non-increasing beyond a threshold level `τ; that is,
R(`τ+n)≥ R(`τ+n+1), ∀n ∈ Z+. (6)
3. The Algorithm
We introduce our algorithm in Section 3.1, derive our rate matrix bounds in Section 3.2, and describe a method to obtain the threshold level `τ in Section 3.3.
3.1. Our Algorithm
We propose Algorithm 2 to calculate the steady-state probability vector π(`0) for any desired level `0∈ Z+. Unique features of Algorithm 2 that distinguish it from Algorithm 1 are: (i) it endogenously determines the truncation level `∗, and (ii) it calculates lower and upper bounds, π(`0) and π(`0), such that kπ(`0)− π(`0)kmax≤ s, for any desired error tolerance s.
Algorithm 2: Computing bounds on π(`0).
1 Input: `0, `τ, p∗, and vectors = (r, s) and θ = (λ1, λ2, µ1, µ2, γ1, γ2, s)
2 initialization: ` = 0, x(`0)= 0, x(`0)= (∞, ..., ∞), c = c1= 0, c2= ∞, and c = c1+ c2
3 calculate R(0) and R(0) // Call Algorithm 3
4 solve x(0)
Q(0)↔+ R(0)Q(1)↓
= 0 for x(0) and x(0)
Q(0)↔+ R(0)Q(1)↓
= 0 for x(0) // Call Algorithm 6
do
5 x(`+1)= x(`) R(`) and x(`+1)= x(`) R(`)
6 calculate R(`+1) and R(`+1) // Call Algorithm 3
7 if ` ≥ `τ and σ R(`+1)
< 1 then
8 c2= x(`+1)
I − R(`+1)−1 1
9 end
10 update normalizing constants: c = c + x(`)1, c1= c1+ x(`)1, and c = c1+ c2
11 π(`0)= x(`0)/c and π(`0)= x(`0)/c
12 ` = ` + 1
13 while kπ(`0)− π(`0)kmax> s
14 return: `∗= ` − 1, π(`0) and π(`0).
Algorithm 2 requires lower and upper bounds, R(`) and R(`), for the rate matrix R(`). We calculate these bounds by exploiting the probabilistic interpretation of the rate matrix such that kR(`)− R(`)kmax≤ r, for any desired error tolerance r, as discussed in Section 3.2.
Algorithm 2 uses the rate matrix bounds to calculate lower and upper bounds, x(`) and x(`), respectively, for the unnormalized stationary probability x(`) by solving equations similar to (5b), when ` = 0, and equations similar to (5a), when ` > 0. Given x(`) and x(`), Algorithm 2 proceeds with calculating lower and upper bounds, c and c, for the normalizing constant c. While c is updated in each iteration ` by accumulating the recently calculated x(`), the calculation of c is more involved and has two requirements: (i) the level ` should be above the threshold level `τ (see (6)), and (ii) the spectral radius should satisfy σ
R(`+1)
< 1, which allows us to bound the tail (i.e., levels ` + 1 and above) probability. These requirements are analogous to bounding the tail probability of a birth-death process with a geometric series (Ingolfsson and Tang 2012). Algorithm 2 calculates x(`0) and x(`0) only once (when ` = `0), but it keeps updating π(`0) and π(`0) in Line 11, as the normalizing constant bounds get updated.
The most expensive operation in both Algorithms 1 and 2 involves matrix inversion, which is used in approximating the rate matrix in Algorithm 1 (Step 4) and in bounding the rate matrix in Algorithm 2 (Steps 3 and 6). Matrix inversion has complexity O(n3) for a square matrix of size n, if one uses Gaussian elimination. For a fixed truncation level `∗, Algorithm 1
0 0.25 0.5 0.75 1
Probability
0 (0)
1 (0)
2 (0)
3 (0) Upper bound Lower bound
0 0.25 0.5 0.75 1
Probability
0 (0)
1 (0)
2 (0)
3 (0)
0 0.25 0.5 0.75 1
Probability
0 (0)
1 (0)
2 (0)
3
(0) 0
0.25 0.5 0.75 1
Probability
0 (0)
1 (0)
2 (0)
3 (0)
Figure 2 Bounds on π(0) for the M/M/s + M (Pr) system with parameter vector θ = (0.09, 8.1, 1, 10, 10, 10, 1).
runs faster than Algorithm 2, because Algorithm 1 performs the matrix inversion `∗ times on a square matrix of size p∗+ 1, for an overall complexity of O(`∗p∗3), whereas Algorithm 2 performs the matrix inversion `∗ times on a square matrix of size (m − 1)(p∗+ 1), for an overall complexity of O(`∗p∗3m3). The advantage of Algorithm 2, however, is that it calculates the truncation level `∗ endogenously such that the stationary probabilities are guaranteed to be within a desired error tolerance, while `∗ is exogenous in Algorithm 1 and the accuracy of the resulting stationary probabilities is unknown.
Figure 2 illustrates the use of Algorithm 2 to bound π(0) for an M/M/s + M (Pr) sys- tem with parameters θ = (0.09, 8.1, 1, 10, 10, 10, 1) and tolerances p= 10−6, r= 10−10, and
s = 0.01. At Iteration 0, the truncation level `∗ starts at 0 and the bounds are π(0) = (0.000, 0.000, 0.000, 0.000) and π(0)= (0.917, 0.083, 0.001, 0.000). At each subsequent iteration, the truncation level `∗ increases by 1 and the gap between the lower and upper bounds decreases. The iterative process yields `τ= 1. When the algorithm terminates after 3 iterations, we find `∗= 3, π(0)= (0.408, 0.037, 0.000, 0.000) and π(0)= (0.412, 0.037, 0.000, 0.000).
3.2. Rate Matrix Bounds
The probabilistic interpretation of the (p, p0)th element of R(`) is the expected sojourn time in state (` + 1, p0) per time unit in (`, p), before returning to level `, given the system starts in (`, p). For a level-independent QBD, Van Leeuwaarden and Winands (2006, p. 82) decompose
the element rpp0 of the rate matrix R based on this probabilistic interpretation as
rpp0= Prpp0E [Np0] qp.
qp0., p, p0∈ H, (7)
where Prpp0 is the probability of moving from (`, p) to (` + 1, p0) without visiting level ` in between; the random variable Np0 is the number of visits to (` + 1, p0) before returning to level
`, given the system visits (` + 1, p0) at least once; and qp./qp0., or equivalently (1/qp0.)/(1/qp.), is the expected sojourn time in (` + 1, p0) per time unit in (`, p).
Adopting the taboo probability notation from Neuts (1981), we use LPr(`
1,p1)(`2,p2)to denote the probability of moving from state (`1, p1) to (`2, p2) without visiting levels in the taboo set L in between. That is, while `1 and `2 may belong to L, the system does not visit any of these levels in between visiting (`1, p1) and (`2, p2). For example, {`}Pr(`,p)(`+1,p0) denotes the probability of moving from state (`, p) to (` + 1, p0) without visiting level ` in between, and {`,`+1}Pr(`,p)(`+1,p) denotes the probability of moving from (`, p) to (` + 1, p) in one step without visiting levels ` and ` + 1 in between (because an LDQBD is skip-free in the level).
We extend (7) for an LDQBD as
rpp(`)0={`}Pr(`,p)(`+1,p0)EhNp(`+1)0
i q(`)p.
q(`+1)p0.
, ` ∈ Z+, p, p0∈ H, (8)
where all quantities in the right-hand side of (8) are level-dependent counterparts of those in (7). In M/M/s + M (Pr) LDQBDs, the only way to move from state (`, p) to a state in level
` + 1 without visiting any states in level `, is through state (` + 1, p) (see Figure 1). Therefore, we can decompose the taboo probability term in (8) as {`}Pr(`,p)(`+1,p0)={`,`+1}Pr(`,p)(`+1,p)×
{`}Pr(`+1,p)(`+1,p0). Using this decomposition, we rewrite (8):
rpp(`)0={`,`+1}Pr(`,p)(`+1,p)×{`}Pr(`+1,p)(`+1,p0)EhNp(`+1)0
i qp.(`) qp(`+1)0.
(9)
= λ2 q(`+1)p0.
{`}Pr(`+1,p)(`+1,p0)EhNp(`+1)0
i, ` ∈ Z+, p, p0∈ H, (10)
where the first term is obtained because the only way to move from (`, p) to (` + 1, p) without visiting levels ` and ` + 1 in between is to transition directly from (`, p) to (` + 1, p), which occurs with probability {`,`+1}Pr(`,p)(`+1,p)= λ2/qp.(`).
Let the random variable Vpp(`+1)0 be the time that the system spends in (` + 1, p0) before visiting level `, given it starts from (` + 1, p). We calculate EhVpp(`+1)0
i as the expected number of times the system visits (` + 1, p0) before visiting level `, given it starts from (` + 1, p), multiplied by the expected sojourn time in (` + 1, p0) for each visit, 1/qp(`+1)0. . That
is, EhVpp(`+1)0
i={`}Pr(`+1,p)(`+1,p0)EhNp(`+1)0
i/qp(`+1)0. . Therefore, (10) is equivalent to:
r(`)pp0= λ2EhVpp(`+1)0
i, ` ∈ Z+, p, p0∈ H. (11)
We build on (11) to calculate r(`)pp0 and r(`)pp0 by under- and overestimating EhVpp(`+1)0 i, respectively. To underestimate, we omit certain sample paths. Specifically, we assume a taboo level ` + m and limit sample paths from state (` + 1, p) to (` + 1, p0) to only those that do not involve levels ` + m, ` + m + 1,. . . , for some positive integer m. Figure 3 illustrates this procedure using ` + 4 as the taboo level: Figure 3(a) displays an included sample path, and Figure 3(b) displays an omitted sample path, for calculating an underestimation of EhV2,3(`+1)i.
To overestimate EhVpp(`+1)0
i, we modify certain sample paths to force them to visit state (` + 1, p0). Specifically, we modify the LDQBD such that the system jumps to (` + 1, p0) with probability 1 from any state in level ` + m. Figure 3(c) displays the sample path for this modified Markov chain that corresponds to the Figure 3(b) sample path.
We are now ready to introduce Algorithm 3 to calculate the rate matrix bounds, R(`) and R(`). This algorithm starts at m = 1 and iteratively increases m by 1 until kR(`)− R(`)kmax≤ r is satisfied. At each iteration, we use the theory of absorbing Markov chains to calculate under- and overestimates, e(`+1)pp0 and e(`+1)pp0 , of EhVpp(`+1)0
i. To calculate e(`+1)pp0 , we modify our LDQBD to a continuous-time absorbing Markov chain J = {Jt, t ∈ R+}, with state space Ω = {(i, j) : ` ≤ i ≤ ` + m and j ∈ H}. Let Ωa= {(i, j) : i = `, ` + m and j ∈ H} denote the set of absorbing states, and Ωca= Ω\Ωa the set of transient states. The transition rates from transient states remain the same as their counterparts in the original system. Therefore, the infinitesimal generator for System J in canonical form (Kao 1996) is:
QJ=
Ωa Ωca Ωa 0 0 Ωca W1 W2
, (12)
1 2 3 4
` `+1 `+2 `+3 `+4
Level
Phase Taboolevel
(a) Path used in e(`+1)2,3 and e(`+1)2,3 calc.
` `+1 `+2 `+3 `+4
Level
Taboolevel
(b) Path omitted in e(`+1)2,3 calc.
` `+1 `+2 `+3 `+4
Level
(c) Path used in e(`+1)2,3 calc.
Figure 3 Sample paths from (` + 1, 2) to (` + 1, 3), given the system never visits ` and m = 4.
1 2 3 4
` `+1 `+2 `+3 `+4
Level
Phase Taboolevel
(a) Figure 3 (a) sample path in System J
` `+1 `+2 `+3 `+4
Level
Taboolevel
(b) Figure 3 (b) sample path in System J
` `+1 `+2 `+3 `+4
Level
Taboolevel
(c) Figure 3 (c) sample path in System J0
Figure 4 Projections of sample paths from Figures 3(a)-3(c) onto Systems J and J0. The symbol • shows an absorbing state.
with lexicographically-ordered rows/columns: Ωa= {(`, 0), ..., (`, p∗), (` + m, 0), ..., (` + m, p∗)}
and Ωca= {(` + 1, 0), ..., (` + 1, p∗), (` + 2, 0), ..., (` + 2, p∗), ..., (` + m − 1, 0), ..., (` + m − 1, p∗)}.
The projection of the sample paths in Figures 3(a)–(b) onto the space of System J is displayed in Figures 4(a)–(b). Initializing System J in the transient state (` + 1, p), we calculate e(`+1)pp0 as the expected sojourn time in the transient state (` + 1, p0) before absorption.
That is, e(`+1)pp0 equals the sojourn time of the original LDQBD in state (` + 1, p0) on sample paths that never visit the taboo level ` + m plus the sojourn time in (` + 1, p0) on sample paths that visit the taboo level, before the first visit to level `.
For System J , the expected sojourn time in state j before absorption, given the system starts in i is the (i, j)th element of the (m − 1)(p∗+ 1) × (m − 1)(p∗+ 1) matrix T := −W2−1 (Kao 1996, p. 257). Therefore, Algorithm 3 constructs E(`+1) =e(`+1)pp0
using the top-left
Algorithm 3: Computing bounds on R(`).
1 Input: `, p∗, r, and θ = (λ1, λ2, µ1, µ2, γ1, γ2, s)
2 initialization: m = 1, R(`)= 0 and R(`)= ∞,
3 while kR(`)− R(`)kmax> r do
4 m = m + 1,
5 construct the infinitesimal generator matrix for System J as in (12),
6 calculate T = (tpp0) = −W2−1 and K = (kpp0) = −W2−1W1.
7 construct E(`+1)= e(`+1)pp0
= tpp0, ∀p, p0∈ H,
8 construct F(`+1)= fpp(`+1)0
= kpp0, ∀p, p0∈ H,
9 construct G(`+1)= gpp(`+1)0
= kpj, ∀p ∈ H, j ∈ {p∗+ 2, ..., 2(p∗+ 1)},
10 construct ∆(`+1)= δ(`+1)pp0
, such that δ(`+1)pp0 =
e(`+1) p0 p0 g(`+1)
p0 . f(`+1)
p0 .
p = p0 δ(`+1)p0p0 gp.(`+1) p 6= p0
, ∀p, p0∈ H,
E(`+1)= E(`+1)+ ∆(`+1),
11 update R(`)= λ2E(`+1) and R(`)= λ2E(`+1),
12 end
13 return: R(`) andR(`).
(p∗+ 1) × (p∗+ 1) submatrix of T , whose (p, p0) element corresponds to the expected sojourn time in state (` + 1, p0) before absorption, given the system started in (` + 1, p), for all p, p0∈ H.
To calculate e(`+1)pp0 , we modify System J to System J0 such that states at level ` + m are transient and the transition rate from these states to state (` + 1, p0) equals ∞. Initializing System J0 at transient state (` + 1, p), we calculate e(`+1)pp0 as the expected sojourn time in the transient state (` + 1, p0) before system J0 hits one of absorbing states. That is, e(`+1)pp0
equals the sojourn time of the original LDQBD in state (` + 1, p0) on sample paths that never visit state (` + 1, p0) plus the sojourn time in this state on sample paths that visit the taboo level ` + m and immediately transition to state (` + 1, p0). The projection of the sample path displayed in Figure 3(c) onto the space of System J0 is displayed in Figure 4(c).
Given that System J0 starts from (` + 1, p), Algorithm 3 calculates e(`+1)pp0 as a function of e(`+1)pp0 and whether the counterpart System J would get absorbed to state (`, p0), denoted by fpp(`+1)0 , or to state (` + m, p0), denoted by gpp(`+1)0 , for p, p0∈ H. For System J, the probability that the system gets absorbed to state k, given that it starts from i, is the (i, k)th element of the (m − 1)(p∗+ 1) × 2(p∗+ 1) matrix K := −W2−1W1 (Kao 1996, p. 257) . Algorithm 3 constructs F(`+1)=fpp(`+1)0
and G(`+1)=g(`+1)pp0
using the top-left and top-right blocks of size (p∗+ 1) × (p∗+ 1) of K, respectively. As we formally establish in Proposition 1, we use the matrices F(`+1) and G(`+1) to calculate the matrix ∆(`+1), which in turn is used to calculate the upper bound E(`+1)= E(`+1)+ ∆(`+1).
Algorithm 4: Computing the threshold level `τ.
1 Input: p∗, and the vector θ = (λ1, λ2, µ1, µ2, γ1, γ2, s)
2 initialization: ` = s,
3 d(`)pp0={`,`+1}Pr(`,p)(`+1,p)×{`,`+1}Pr(`+1,p0)(`,p0), ∀p, p0∈ H,
4 d(`+1)pp0 ={`+1,`+2}Pr(`+1,p)(`+2,p)×{`+1,`+2}Pr(`+2,p0)(`+1,p0), ∀p, p0∈ H,
5 construct D(`)= d(`)pp0
and D(`+1)= d(`+1)pp0
,
6 while kD(`)− D(`+1)kmax< 0 do
7 ` = ` + 1
8 d(`+1)pp0 ={`+1,`+2}Pr(`+1,p)(`+2,p)×{`+1,`+2}Pr(`+2,p0)(`+1,p0), ∀p, p0∈ H,
9 construct D(`+1)= d(`+1)pp0
,
10 end
11 `τ= `
The novelty of our method for bounding the rate matrix is in the decomposition in (9), which reduces to (11), and the technique to under and overestimate the EhV(`)i matrix.
Although we present our method for an M/M/s + M (Pr) LDQBD, it can be applied to any other LDQBD by adjusting (9) and (11), and applying the same technique to bound EhV(`)i. To do this, one needs to rewrite (9) and (11) by considering all possible paths to directly move from (`, p) to a state in level ` + 1 without visiting any states in level ` in between:
r(`)pp0 =X
j∈H
{`,`+1}Pr(`,p)(`+1,j)×{`}Pr(`+1,j)(`+1,p0)EhNp(`+1)0
i qp.(`) q(`+1)p0.
(13)
=X
j∈H
q(`)↑p,j EhVjp(`+1)0
i, ` ∈ Z+, p, p0∈ H. (14)
Note that qp,j(`)↑={`,`+1}Pr(`,p)(`+1,j)= 0, when j 6= p, for M/M/s + M (Pr) LDQBDs.
3.3. The Threshold Level, `τ
One of the inputs for Algorithm 2 is the threshold level `τ, beyond which (6) holds. We exploit the structure of M/M/s + M (Pr) LDQBDs to establish a sufficient condition in Proposition 3, which guarantees that (6) holds. The sufficient condition (15) states that the one-step probability of moving from (`τ, p) to (`τ + 1, p) multiplied by the one-step probability of moving from (`τ+ 1, p0) to (`τ, p0) is non-increasing in `τ, for all p, p0∈ H. We use Algorithm 4 to iteratively increase ` until (15) is satisfied. The sufficient condition is that (15) holds for all pairs of phases, p, p0∈ H. For ease of exposition, we present Algorithm 4 assuming that for each level, (15) is checked for all pairs, but the algorithm can be made more efficient by iterating to the next level as soon as a violating pair of phases is found.
