Fluid Models for Production-Inventory Systems

Fluid Models for Production-Inventory Systems

by Keqi Yan

A dissertation submitted to the faculty of the University of North Carolina at Chapel Hill in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Statistics and Operations Research.

Chapel Hill 2006

Approved by Advisor: Vidyadhar G. Kulkarni Reader: Amarjit Budhiraja Reader: Tugrul Sanli

Reader: Jayashankar M. Swaminathan Reader: Paul H. Zipkin

c

2006

Keqi Yan

ABSTRACT

Keqi Yan: Fluid Models for Production-Inventory Systems (Under the direction of Professor Vidyadhar G. Kulkarni)

We consider a single stage production-inventory system whose production and de-mand rates are modulated by a finite state Markov chain called the environment. Supplementary orders can be placed from external suppliers when needed. We model this system by a fluid-flow system and derive the limiting distribution of the bivariate process (fluid level, environment state). We present a stochastic decomposition prop-erty for this fluid model and hence prove that the classical deterministic Economic-Order-Quantity (EOQ) policy is still optimal in this stochastic environment under certain assumptions.

We extend the results to more general models:

1. When backlogging is allowed, we investigate the optimal repoint / order-quantity (r, q) policy. We prove that for a given order order-quantityq, the optimal reorder pointr∗(q) can be explicitly given by the well-known newsboy solution. We also show that in a special case the optimality of the deterministic EOQ policy with backlogging holds.

2. When the order quantity can be environment-dependent, we derive the limiting distribution and then calculate the optimal order quantity for each ordering state.

sequentially, in parallel, or the leadtimes have different distributions depending on the number of outstanding orders. Assuming there exists an upper limitN for the number of outstanding orders, this model generalizes the emergency-supply model, selective lost-sales model, and can also be an approximation of infinite-supplier model when N is large enough or the probability that there are N outstanding orders is small. We derive the limiting distribution and the optimal (r, q) policy. We prove that for a given q, the optimality of the newsboy solution for r still holds. We also illustrate numerically how to calculate the optimalN∗ which balances the backlogging cost and emergency-supply / lost-sale cost.

ACKNOWLEDGEMENTS

I have waited for a long time for this chance to express my deepest gratitude to my advisor, Professor Vidyadhar G. Kulkarni. He always earns the respect and admiration of all his students, and for me, he is the hero in my four years’ journey in the pursuit of the Ph.D. at Chapel Hill. His impact on my attitude in research and philosophy about the world will benefit the reminder of my life.

I would also like to thank Professor Paul Zipkin for introducing me to the field of inventory management. During the process of finishing this dissertation, he gave me priceless suggestions and always kindly encouraged me like a father figure. Professor Jayashankar M. Swaminathan’s teaching opened a wide door to the supply chain area. With Dr. Tugrul Sanli, I worked on a demonstration of the software Inventory Replenishment Planning, which was my first exposure to the concept of inventory control and spurred my interest in this area. I am also very thankful to Professor Amarjit Budhiraja for his inspiring questions and suggestions.

Special thanks to my supervisors and friends at SAS Institute from where I have received tuition support for three years, and have gained valuable experience in imple-menting operations research methodologies in software and solving real-world prob-lems. I really enjoyed and have learned so much from these past years’ internship there. It was an inseparable part of my student life in North Carolina.

Of course I would also like to thank my parents Yan Yuchi, He Daying and my sister Yan Shuli. However, I know that words are redundant here, in comparison to their unconditional love and support, which are far beyond any language in the world.

CONTENTS

LIST OF FIGURES . . . xi

LIST OF SYMBOLS . . . xii

1 Introduction 1 1.1 Stochastic Fluid Models . . . 2

1.2 Stochastic Inventory Control Problems . . . 3

1.2.1 Basic EOQ Model . . . 3

1.2.2 Backlogging EOQ Model . . . 4

1.2.3 Environment-Dependent Order Quantities . . . 5

1.2.4 Stochastic Leadtimes . . . 7

2 Fluid Model 9 2.1 Introduction . . . 9

2.2 The Standard Fluid Model . . . 11

2.4 Differential Equations for the Limiting Distribution . . . 16

2.5 Solution to the Differential Equations . . . 19

2.5.1 Case of Distinct Eigenvalues . . . 20

2.5.2 Case of Repeated Eigenvalues . . . 22

2.6 A Special Case: A=I . . . 23

2.6.1 Stochastic Decomposition Property . . . 24

2.6.1.1 Laplace Stieltjes Transform Method . . . 24

2.6.1.2 Sample Path Method . . . 26

2.6.2 Uniform Limiting Distribution . . . 34

2.7 Examples . . . 35

2.7.1 A Two-State Example . . . 35

2.7.2 A Machine Shop Example . . . 38

3 A Basic Production-Inventory Model 40 3.1 Introduction . . . 40

3.2 The Model . . . 42

3.3 Optimal Order Quantity . . . 43

3.3.1 Stochastic EOQ Theorem . . . 44

3.4 Inventory Model with Backlogging . . . 49

3.4.1 Cost Rate Calculation of the (r, q) Policy . . . 50

3.4.2 Optimal (r, q) Policy . . . 52

3.4.2.1 Newsboy Solution for the Optimal r for a Given q . 52 3.4.2.2 Stochastic EOQ Policy with Backlogging . . . 53

3.4.2.3 A Numerical Example . . . 54

4 Environment-Dependent Order Quantities 58 4.1 Introduction . . . 58

4.2 Piecewise Function Method . . . 60

4.3 Sample Path Decomposition Method . . . 64

4.4 Laplace-Stieltjes Transform Method . . . 71

4.5 The Cost Model . . . 76

4.6 A Numerical Example . . . 77

5 Stochastic Leadtimes 80 5.1 Introduction . . . 80

5.2 Serial Processing System . . . 84

5.3 Parallel Processing System . . . 87

5.5 Selective Lost–Sale Model . . . 91

5.6 The Cost Model . . . 93

5.6.1 Cost Rate Calculation . . . 93

5.6.2 Newsboy Solution for the Optimal Reorder Point . . . 97

5.7 A Numerical Example . . . 99

5.7.1 Limiting Distribution . . . 99

5.7.2 Optimal Ordering Policy . . . 102

5.7.3 Optimal Production Rate . . . 104

5.7.4 Sensitivity Analysis . . . 104

6 Conclusions and Future Research 111 6.1 Conclusions . . . 111

6.2 Future Research . . . 114

6.2.1 Model with Semi-Markov Process as Background process . . . 114

6.2.2 Environment-Dependent Order Quantities and Reorder Points 115 6.2.3 Numerically Stable Methods for the Stochastic Leadtime Model in Serial Processing System . . . 115

6.2.4 Environment-Dependent Ordering Policies with Stochastic lead-times . . . 115

LIST OF FIGURES

2.1 A sample path of the (buffer level, environment state) process. . . 10

2.2 Decomposition of theX(t) process. . . 27

2.3 Correspondence of the processesX(t), Z(t), X1(t),Y0(t) and Z0(t). . 30

2.4 Correspondence of the processesX(t), Z(t), X1(t),Y1(t) and Z1(t). . 32

2.5 Limiting distribution when r > d. . . 37

2.6 Limiting distribution when r < d. . . 38

2.7 The steady-state ccdf. . . 39

2.8 The steady-state pdf. . . 39

3.1 The optimal order quantity vs. production rate. . . 46

3.2 The minimum total cost and the optimal production rate. . . 48

3.3 The inventory level process when allowing backlogging. . . 49

3.4 The optimal order quantity vs. production rate. . . 54

3.5 The optimal reorder point vs. production rate. . . 55

3.6 The optimal order-up-to level q∗+r∗ vs. production rate. . . 56

3.7 The minimum cost vs. production rate. . . 57

4.1 A sample path of X(t) and Z(t) with environment-dependent order quantities. . . 59

4.2 Piecewise function method. . . 61

4.3 Sample path decomposition method. . . 65

4.4 The optimal order quantities vs. production rate. . . 78

5.1 Sample paths ofP(t) and X(t) with stochastic leadtimes. . . 81

5.2 The steady-state ccdf of the ¯P(t) process. . . 99

5.3 The steady-state pdf of the ¯P(t) process. . . 100

5.4 The steady-state ccdf of X(t) and ¯P(t). . . 101

5.5 The steady-state pdf ofX(t) and ¯P(t). . . 101

5.6 The optimal order quantity vs. production rate (varying n). . . 102

5.7 The optimal reorder point vs. production rate (varying n). . . 103

5.8 The minimal cost vs. production rate (varying n). . . 104

5.9 The optimal order quantity vs. production rate (varying ν). . . 105

5.10 The optimal reorder point vs. production rate (varying ν). . . 105

5.11 The minimum cost vs. production rate (varying ν). . . 106

5.12 The optimal order quantity vs. production rate (varying N). . . 107

5.13 The optimal reorder point vs. production rate (varying N). . . 108

5.14 The probability that there are N outstanding orders (varyingN). . . 108

LIST OF SYMBOLS

aj Equation (2.17) and (4.27), Theorem 4.2, Theorem 4.6.

a A row vector of aj’s in Equation (4.27).

akj Defined in Equation (5.15) and (5.22) .

A Transition probability matrix [αij], Section 2.3.

A(i) _{Defined in Theorem 4.3.}

¯

A Defined in Equation (5.26).

b Backorder penalty-cost rate, Section 3.4.1. cb Steady-state backlogging cost rate.

ch Steady-state holding cost rate.

co Steady-state ordering cost rate.

cp Steady-state production cost rate.

cj Coefficients in Equation (2.16).

c(_ki) Defined in Theorem 4.2. d Demand rate in Section 2.7.1.

di Demand rate when the environment process is in state i, Section

2.7.2. ˜

D(s) Defined in Theorem 4.6. e e= [1, ...,1]t_.

E(x) Expectation of the random variable x. ˜

Eii(s) Defined in Equation (4.31).

f(j, x) Defined in Equation (2.33).

Fj(x) Limiting cdf of the inventory level process at statej, Equation (2.21).

˜

Fj(s) LST of Fj(x), Equation (2.22).

˜

g(i, x) Defined in Equation (2.36) . G(x) G(x) = [G1(x), ..., Gn(x)]. G0(x) G0(x) =hdG1(x) dx , ..., dGn(x) dx i . Gj(t, x) Defined in Equation (2.11). Gj(x) Defined in Equation (2.12). G(_ji)(x) Defined in Section 4.3. G(i)_{(x) Defined in Section 4.3.} ¯ G(_ji)(x) Defined in Section 5.1 . ¯ G(i)_{(x) Defined in Section 5.1 .} ¯ G(x) Defined in Section 5.1 .

h Inventory holding-cost rate, Section 3.1.

Hj(x) Limiting Cumulative distribution function of the fluid level in state

j, in standard fluid model without jumps, Section 2.2. H(x) [H1(x), ..., Hn(x)], Section 2.2. H0(x) h dH1(x) dx , ..., dHn(x) dx i , Section 2.2. I(i) Defined in Equation (4.2).

k Fixed set-up cost to place an order, Section 3.1.

k1 Fixed set-up cost to place an order from a regular supplier, Section

5.6.

k2 Fixed set-up cost to place an order from an emergency supplier,

Section 5.6.

L Defined in Equation (4.30).

m+ Number of background states with positive input rate.

m0 Number of background states with zero input rate.

m− Number of background states with negative input rate. m Number of background states with nonzero input rate.

M_ii(j) Defined in Equation (4.39). M(j) _{A diagonal matrix diag(M}(j)

ii ).

n Number of the states of the environment processes.

N Upper limit of the number of outstanding orders, Section 5.1. O(t) Number of outstanding orders at timet, Section 5.1.

p(i) _{Defined in Section 4.3.}

pik Defined in Section 4.3.

p(k)_{(x) Defined in Section 4.3.}

pik(x) Defined in Section 4.3.

P, P = [pik], Section 4.3.

P(t) Inventory position at timet. ¯

P(t) Defined in Equation (5.1).

p1 Purchasing cost rate, Section 3.3.

Purchasing cost rate from a regular supplier, Section 5.6.1. p2 Production cost rate, Section 3.3.

Purchasing cost rate from the emergency supplier, Section 5.6.1. p3 Production cost rate, Section 5.6.1.

q Order quantity.

qi Order quantity when the order is place in state i.

qij Transition rate of the environment process from statei to statej.

ˆ

qij Defined in Equation (2.30).

Q Q= [qij].

¯

Q Defined in Equation (5.4), (5.12) and (5.19). r Reorder point.

¯

Ri Net input rate when the environment process is in state i. Ri =

ri−di.

Si i-th order epoch, Section 2.6.1.2.

t Time variable.

T First passage timeT = inf{t≥0 :X(t) = 0}, Equation (4.10). T1 Defined in Section 2.6.1.2.

T2 Defined in Section 2.6.1.2.

T2n+1 Defined in Section 2.6.1.2.

T2n+2 Defined in Section 2.6.1.2.

Tj(x) Steady-state complementary cdf of the inventory level in the

back-logging model, Equation (3.8). T(x) Vector ofTj(x)’s, Equation (3.8).

u Production rate of one machine, Section 2.7.2. x Inventory level variable.

X(t) Inventory level at timet. X1(t) Defined in Section 2.6.1.2.

X2(t) Defined in Section 2.6.1.2.

Y Defined in Section 5.6.2. Y0(t) Defined in Section 2.6.1.2.

Y1,n Defined in Section 2.6.1.2.

Z(t) State of the environment process at timet. Z0(t) Defined in Section 2.6.1.2.

αij Defined in Section 2.3.

β Defined in Theorem 2.3.

β(i) _β(i) _=G0_(0)RI(i)_{, Theorem 4.1.}

δ A small positive number, Section 2.4. δij δij = 1 ifi=j, and 0 otherwise.

∆ Net demand rate in steady state, Section 3.2. ηkj Defined in Equation (2.31).

θ Defined in Section 2.7.1.

λ Repair rate of failed machine, Section 2.7. λi i-th generalized eigenvalue, Equation (2.5).

λ(_ik) Defined in Section 5.3, and Equation (5.20) . µ Failure rate of one machine, Section 2.7. ν Leadtime distribution parameter, Section 5.1. π Defined in Equation (2.1).

ˆ

πi Defined in Equation (2.32).

π(j, x) Defined in Equation (2.35) .

τ(j, x) Expected sojourn time of the SMP in state (j, x), Section 2.6.1.2. τj τj =τ(j, x), ∀x, Section 2.6.1.2.

φi The row vector (eigenvector) corresponding to λi such that

φi(λiR−Q) = 0.

φ0 A constant row vector in the expression of G(x), Theorem 2.4.

φ(₀i) Defined in Theorem 4.2.

φ(_ik) Defined in Section 5.3, and Equation (5.20) . Φ Defined in Equation (4.4).

ψj (λjR−Q)ψj = 0, Section 4.4.

Ψ Ψ = R−1_Φ−1_{, Section 4.4.}

Ω State space of the environment process. Ω+ Ω+ ={i∈Ω :Ri ≥0}.

Chapter 1

Introduction

In this thesis we study a type of production-inventory models that can be seen as a stochastic fluid-flow system. We consider a single product, single location prob-lem. The system has production, demand, and external supply. As the environment evolves over time, the production and demand rates are piecewise constant functions determined by the exogenous environment process. When the production rate ex-ceeds the demand rate, the inventory increases, and when the demand rate exex-ceeds the production rate, it decreases. When needed, replenishment orders can be placed from external suppliers. The inventory under continuous review thus can be viewed as a fluid process that fluctuates according to the evolution of the underlying back-ground process. We assume the external environment undergoes recurring changes in a stochastic fashion, and may be modeled as Markovian. For example, production rates and demand rates change due to weather, economy, competition, seasonal pro-motion, customer status, and forecasting, etc. Some other example are as described in Mitra (1988) where the author studied a producer and consumer problem in a machine shop where the production rate changes according to the number of working machines. There are costs to hold products in inventory, to backlog unsatisfied orders,

to purchase and to produce. There is also a fixed set-up cost every time an order is placed with an external supplier. Our objective is to find an optimal ordering and production policies that minimizes the long-run average cost.

1.1

Stochastic Fluid Models

First in Chapter 2 we study a fluid model to establish the fundamental theory for the production-inventory system. We view the inventory level under continuous review as a fluid level process. When the buffer is empty the fluid level jumps to a predetermined levelq instantaneously, and at the same time the environment state jumps to another state with a given probability (it may stay unchanged). Between two consecutive jumps the background process is a continuous time Markov chain. At the jump epoch the environment process jumps according to a transition matrix A.

We first derive the stability condition for this system and then derive a set of first order non-homogeneous linear differential equations to describe the limiting behav-ior of the bivariate (buffer level, environment state) process. We also determine the boundary conditions and give explicit solutions to the differential equations. Partic-ularly for a special caseA =I, we use two parallel methods to obtain an interesting stochastic decomposition property: in steady state, the buffer content in the fluid model with jumps is the sum of two independent random variables, one of which has a uniform distribution over [0, q], and the other is the steady-state buffer level in the standard fluid model without jumps. We also consider a more specific case where the fluid input rate is always negative over all the environment states. In this case the fluid level has uniform distribution in steady state, and is independent of the environment state.

change in the buffer content. Under that assumption, the limiting joint distribution of the (buffer level, environment state) process is computed as a solution of a set of ordinary differential equations in terms of the eigenvalues and eigenvectors of the underlying system. There are also studies about fluid models where instantaneous jumps occur when the environment state changes and the size of the jump depends on the state of the environment (see Kulkarni, Tzenova and Adan (2005), Miyazawa and Takada (2002) and Sengupta (1989)). Another related model is the so called

clearing system (see El-Taha (2002), Serfozo and Stidham (1978) and Whitt (1981)).

A clearing system can be regarded as the reverse of our model. In a clearing system the fluid process jumps to zero when it reaches a certain positive level. However, there is no explicit environment process in these models. The paper that comes closest to our analysis is Berman, Stadje and Perry (2006) where the authors consider a two state CTMC as the environment process. The methodology of their analysis is different with ours.

1.2

Stochastic Inventory Control Problems

Beginning from Chapter 3, we study the production-inventory problem using the theories developed in Chapter 2.

1.2.1

Basic EOQ Model

In Chapter 3 we start from the basic model where there is no backlogging and zero leadtime, i.e., when the inventory on hand is zero, a supplementary order is placed and arrives instantaneously, and the order size q is independent of the environment state when the order is placed.

In a deterministic setting with constant demand rate, the classical

Economic-Order-Quantity (EOQ) model describes the trade-off between the constant set-up

cost and the variable holding cost. The earliest work on this is Harris (1913). For a modern review of the determinist models, see Zipkin (2000). In this thesis, we estab-lish the stochastic EOQ theorem that shows in a CTMC environment the standard deterministic EOQ formula remains optimal if we replace deterministic demand rate by the expected net demand rate in steady state.

In addition to the ordering policy, we also consider the optimal production policy: choose the optimal production capacity that achieves the best combination of out-sourcing and inhouse-production. We show this mainly with numerical results. Inter-estingly, the optimal policy does not suggest always depending on inhouse-production, even if the production cost is less than the outsourcing price.

1.2.2

Backlogging EOQ Model

Later in Chapter 3, from Section 3.4 we extend the basic model to allow backlogging: an external order is not placed until the inventory level reaches the preset reorder point r. We derive the optimal ordering-production policy, which achieves the trade-off point of production cost, fixed ordering cost, holding cost and backlogging cost. Particularly, when production is always less than the demand rate (for example, the system does not make its own product), we prove that the optimality of the deterministic EOQ formula with backlogging still holds in this stochastic environment if one replaces the deterministic demand rate by the expected net demand rate in steady state.

In the literature, dynamic control of inventory systems have been classified as periodic review models and continuous review models. The continuous review model

can be further classified according to the demand and production are discrete or continuous.

(1) Discrete demands and production: Poisson demands in the continuous review models are studied in depth, see Scarf (1958), Karlin and Scarf (1958), Galliher et al. (1959), and Morse (1958). These early papers are reviewed in Scarf (1963). Poisson demand is generalized by Finch (1961), Rubalskiy (1972a,b) and Sivazlian (1974) to unit demands arriving at epochs following a renewal process. Song and Zipkin (1993) considered the case of Markov modulated Poisson demands.

(2) Continuous fluid models: Berman and Perry (2004) studied a fluid model where the production and demand rates depend on the inventory level. Browne and Zipkin (1991) studied a model with continuous demand driven by a Markov process, which can be regarded as a special case of the model in this thesis.

To our knowledge, although there are papers studying similar problems, none includes the result about the explicitStochastic EOQ theorem presented in this thesis.

1.2.3

Environment-Dependent Order Quantities

In Chapter 4 we relax the assumption that the order quantity is predetermined, i.e., the order quantity is allowed to depend upon the environmental state when the order is placed. For example, if we can observe the environmental state and can base our inventory replenishment decisions on that information, the policy that allows the order quantities to depend on the state of the environment is certainly no worse than the simple reorder-point/order-quantity policy.

(1) Consider appropriate non-overlapping intervals of the inventory level and within each interval derive the differential equations following the methodology of Chapter 2. Then the final limiting distribution is a piecewise function con-sisting of the functions derived in all these intervals.

(2) Decompose the sample path of the inventory level into different cycles and re-duce this problem to the basic fluid model of Chapter 2 in individual cycles. Then the overall limiting distribution is a weighted average of the limiting dis-tribution functions in all cycles.

(3) Consider individual intervals of the inventory level as in (1), but use Laplace-Stieltjes transforms instead of solving differential equations to obtain the limit-ing distribution function.

Then based on the limiting distribution of the inventory level, we derive the long-run average cost and hence determine the optimal environment-dependent order quan-tities.

In the literature Berman, Stadje and Perry (2006) studied a similar model with a two-state random environment. They consider order quantities that depend on the state of the environmental state and derive the optimal order quantities to maximize the system revenue. However, their calculation of optimal order quantities is based on the explicit results of the steady-state distribution of this two-state system. When the background has more than two states, their method becomes impractical.

The general fluid EOQ models with multiple order quantities studied in this thesis seem to be new.

1.2.4

Stochastic Leadtimes

In Chapter 5 we extend the model further to allow stochastic leadtimes. Three order processing fashions are considered:

(1) Orders are processed sequentially, and hence orders never cross in time. Inter-arrival times between orders are i.i.d. exponential.

(2) Orders are processed in parallel fashion, and leadtimes are i.i.d. exponential random variables. So orders can cross in time.

(3) Inter-arrival times of the outstanding orders have exponential distributions whose parameters depends on the number of outstanding orders. This gen-eralizes the previous two cases.

We assume there exists an upper limit N of the number of outstanding orders. When there areN outstanding orders and the inventory position decreases to the re-order point again, we either obtain an “emergency re-order” instantaneously with higher ordering costs, or lose sales with the penalty costs. If the limiting probability that the number of outstanding orders isN is very small, this provides a good approximation to the models with no upper limit on the number of outstanding orders

We derive the optimal ordering-production policy which minimizes the sum of the production cost, fixed ordering cost, holding cost, backlogging cost, and emergency ordering cost (or lost-sale penalty). Minimum cost and limiting distribution under the optimal policy are also calculated.

In the literature, there is a sizeable body of work on inventory systems with stochastic leadtimes and Markov modulated demands. However, most of this litera-ture is concerned with Markov modulated Poisson process models of demands. An

extensive review of this literature is given in Zipkin (2000). As far as we know, there is very little work on Markov modulated fluid models in the context of the production-inventory systems. One relevant work is that of Browne and Zipkin (1991), where the authors assume continuous stochastic demand, but no production.

Chapter 2

Fluid Model

2.1

Introduction

In this chapter we study a stochastic fluid-flow system consisting of a single infinite capacity buffer. The buffer content increases or decreases according to a fluid-flow rate modulated by an environment which is a stochastic process with finite state space. Whenever the buffer is empty, it is refilled to a predetermined level instantaneously, and at the same time the environment state jumps to another state with a given probability (it may stay unchanged). Figure 2.1 illustrates a sample path of the (buffer level, environment state) process. Our primary motivation for considering this model is to provide fundamental theories to study a production-inventory system modulated by a Markovian environment. For example, the fluid process can be viewed as the inventory level under continuous review. The environment process represents the background state, for example, production or sales seasons. A jump in the fluid level represents an external order placement or order arrival, and the transition of the background state at the jump point can be a result of repairs of production facility, etc.

Figure 2.1: A sample path of the (buffer level, environment state) process. The outline of this chapter is as follows. In section 2.2, we present some prelimi-nary results about the standard fluid model without jumps. In section 2.3 we describe the model with jumps in detail and derive the stability condition. In section 2.4 we derive a system of first order non-homogeneous linear differential equations for the limiting distribution of the bivariate (buffer level, environment state) process. We also determine the boundary conditions needed to solve those differential equations. In section 2.5, we derive explicit solutions to the differential equations. An interesting stochastic decomposition property is given in Section 2.6 about a special case where the background state does not change at jump epochs: in steady state, the buffer con-tent in the fluid model with jumps is the sum of two independent random variables, one of which has a uniform distribution over [0, q], and the other is the steady-state buffer level in the standard fluid model without jumps. We also consider a more specific case where the fluid input rate is always negative over all the environment states. In this case the fluid level has uniform distribution in steady state, and is independent of the environment state. In section 2.7, we illustrate our methodology

with an analytic example as well as a numerical one.

2.2

The Standard Fluid Model

In this section, we present some preliminary results about the standard fluid model with infinite capacity buffer. See the survey paper Kulkarni (1997) for an extensive overview of the research in this area. Let X(t) be the fluid level in the buffer at time t. The rate of change of the fluid level is modulated by a continuous time Markov chain {Z(t), t ≥ 0} on a finite state space Ω = {1,2, ..., n} with generator matrix Q = [qij]. As long as Z(t) is in state i, the fluid level process {X(t), t ≥ 0} changes

at rate Ri. Note that Ri may be either negative or positive. Let π = [π1, π2, . . . , πn]

be the limiting distribution of the{Z(t), t ≥0} process, i.e., π is the unique solution to πQ= 0, n P i=1 πi = 1. (2.1)

The system is stable if and only if the expected input rate is negative, i.e.,

n

X

i=1

πiRi <0.

Let R= diag(R1, ..., Rn) be the diagonal n×n matrix with the input rate Ri as the

ith entry on the diagonal. Let e= [1, ...,1]t _{be an n×1 column vector of ones. Then}

the stability condition can be written in matrix form as follows

When the stability condition (2.2) holds, the following limits exist: Hj(x) = lim t→∞P{X(t)≤x, Z(t) =j}, x≥0, j ∈Ω. Let H(x) = [H1(x), ..., Hn(x)], and H0(x) = dH1(x) dx , ..., dHn(x) dx .

The next theorem gives the differential equations satisfied byH(x).

Theorem 2.1. Assume the stability condition (2.2) holds. The vector H(x)satisfies

H0(x)R=H(x)Q, x≥0. (2.3)

The boundary conditions are given by

Hj(0) = 0, ∀j :Rj >0, (2.4a)

H(∞)e= 1. (2.4b)

Let (λ, φ) be a generalized (eigenvalue, eigenvector) pair that solves

Let

Ω+={i∈Ω :Ri >0}, (2.6)

Ω0 ={i∈Ω :Ri = 0}, (2.7)

Ω−={i∈Ω :Ri <0}, (2.8)

andm+=|Ω+|,m0 =|Ω0|andm− =|Ω−|. It is known that the number of eigenvalues

that satisfy Equation (2.5) is m = m+ +m− (counting multiplicities). When the

stability condition holds, one eigenvalue is 0,m+ have negative real part, andm−−1

have positive real part. We index the eigenvalues so that λ1, ..., λm+ have negative

real parts, λm++1 = 0, and λm++2, ..., λm have positive real parts. It is easy to see

that φm++1 =π is a valid eigenvector corresponding to the eigenvalue 0.

When the eigenvalues are all distinct, the solution to the differential equations in Theorem 2.1 is given by H(x) = m+ X i=1 aieλixφi+π, if x >0,

where the coefficients a1, ..., am+ are given by the unique solution to the following

system of m+ linear equations:

m+

X

i=1

aiφij +πj = 0, ∀j ∈Ω+,

where φij is the j-th element in φi. Let

˜ Hj(s) =

Z ∞

0

be the Laplace-Stieltjes transform (LST) of Hj(x) and

˜

H(s) =hH˜1(s),H˜2(s), ...,H˜n(s)

i .

Taking transforms of (2.3), and noticing thatH(x) has a jump at 0 of sizeH(0), and a density H0(x) for x >0, we get

˜

H(s) =sH(0)R(sR−Q)−1. (2.9)

It follows that there is a unique vector H(0) satisfying conditions (2.4a) and (2.4b) that makes ˜H(s) a valid LST of a vector of random variables. We shall use this fact in deriving results in section 2.6.

2.3

The Fluid Model with Jumps

Now we describe a fluid-flow model with infinite capacity buffer that we analyze in this chapter. As before, let X(t) be the fluid level in the buffer at time t. The rate of change of the fluid level is modulated by a stochastic process {Z(t), t ≥ 0} on a finite state space Ω ={1,2, ..., n}. As long asZ(t) is in state i, the fluid level process

{X(t), t≥0}changes at rateRi. WhenX(t) reaches zero it jumps to a predetermined

level q instantaneously. Let S0 = 0 and Sk be the kth jump time. We assume that

over (Sk, Sk+1) the process {Z(t), t ∈ (Sk, Sk+1)} behaves as an irreducible CTMC

on Ω with generator matrix Q= [qij]. Furthermore, when the {X(t), t ≥ 0} process

jumps at timeSk, the{Z(t), t ≥0} process changes instantaneously with probability

αij defined as follows:

Let

A= [αij].

It is clear that {(X(t), Z(t)), t ≥ 0} is a bivariate Markov process. Next we derive the condition when this process is stable, i.e., it has a limiting distribution. Let π be as in Equation (2.1). Note that π is not the limiting distribution of Z unless A=I.

Theorem 2.2. The process {(X(t), Z(t)), t ≥0} is stable if and only if

n

X

i=1

πiRi <0. (2.10)

Proof. LetSk be the k-th jump epoch in the {X(t), t≥0}process, with S0 = 0.

Let Zk = Z(Sk+). It is easy to see that {Zk, k ≥ 0} is a DTMC on state space Ω.

SinceQis assumed to be irreducible it can be seen that{Zk, k≥0}has a single closed

communication class Ω0 ⊆ Ω that is positive recurrent. Without loss of generality, suppose (X(0), Z(0)) = (q, i) for some i∈Ω0. Let

N = min{k≥0 :Zk =i}.

It is clear that X(SN) = q, Z(SN) = iand that{(X(t), Z(t)), t≥0}is a regenerative

process that regenerates at timeSN. Thus from the theory of the regenerative process

(see Heyman (1982)), the limiting distribution of the {(X(t), Z(t)), t ≥ 0} process exists if E(SN) < ∞. Since Z(0) = i ∈ Ω0, it follows that N is the number of steps

needed by the {Zk, k ≥ 0} process to go from state i to state i. Since Ω0 is finite,

E(N)<∞. Now from Kulkarni (2002), it follows that E(S1|X(0) = x, Z(0) =j)<

∞ if and only if

n

P

i=1

πiRi <0. Now,

E(SN)≤E(N)·max

This proves the theorem.

2.4

Differential Equations for the Limiting

Distri-bution

Let

Gj(t, x) = P{X(t)> x, Z(t) = j}, x≥0, t≥0, j ∈Ω. (2.11)

Assume the stability condition (2.10) holds so that the following limits exist:

Gj(x) = lim

t→∞P{X(t)> x, Z(t) = j}, x≥0, j ∈Ω. (2.12) In this section we show how to compute

G(x) = [G1(x), ..., Gn(x)]. (2.13)

We use the notation

G0(x) = dG1(x) dx , ..., dGn(x) dx .

The next theorem gives the differential equations satisfied byG(x).

Theorem 2.3. Assume the stability condition (2.10) holds. The limiting distribution

G(x) is continuous on [0,∞) and is a piecewise differentiable function on (0, q) and

(q,∞). It satisfies

G0(x)R =G(x)Q+β, 0< x < q, (2.14a) G0(x)R =G(x)Q, x > q, (2.14b)

where the row vector β is given by

β =G0(0)RA.

The boundary conditions are given by

G(∞) = 0, (2.15a)

Gj(q+) = Gj(q−), j /∈Ω0, (2.15b)

G0_j(0) = 0, j ∈Ω+, (2.15c)

G(0)e= 1. (2.15d)

Proof. The differential equations follow from the standard derivation of Chapman Kolmogorov equations for Markov processes. We assume at time 0, (X(0), Z(0)) is in steady-state, i.e., for all state j ∈Ω, P{X(0) > x, Z(0) =j}=Gj(x).

First consider the x < q case. We consider a time interval [0, δ] where δ > 0. During [0, δ], the {Z(t), t ≥ 0} process behaves like a usual CTMC with generator matrix Q if the {X(t), t≥ 0} process does not hit zero. Otherwise the {Z(t), t≥0}

process changes state according to the matrixA and the{X(t), t≥0}process jumps toq. Thus we have Gj(x) = P{X(δ)> x, Z(δ) =j} = n P i=1 P{X(δ)> x, Z(δ) = j|X(0) > x−Riδ, Z(0) =i} ·P{X(0)> x−Riδ, Z(0) =i} + P i∈Ω− P{X(δ)> x, Z(δ) = j|X(0)≤ −Riδ, Z(0) =i} ·P{X(0)≤ −Riδ, Z(0) =i}

When X(0) > x−Riδ, we get

P{X(δ)> x, Z(δ) =j|X(0)> x−Riδ, Z(0) =i}

= P(Z(δ) = j|Z(0) =i) = δij +qijδ+o(δ),

where δij = 1 if i=j, and 0 otherwise. Also note that when X(0)≤ −Riδ,

P{X(δ)> x, Z(δ) = j|X(0)≤ −Riδ, Z(0) =i}=αij +O(δ),

where O(δ) is a function of δ that goes to 0 as δ goes to 0. Using the fact that O(δ)(Gi(0)−Gi(−Riδ)) =o(δ), we get the following:

Gj(x) = n X i=1 (δij +qijδ)·Gi(x−Riδ) + X i∈Ω− αij(Gi(0)−Gi(−Riδ)) +o(δ) = Gj(x−Rjδ) + n X i=1 qijδGi(x−Riδ) + X i∈Ω− αij(Gi(0)−Gi(−Riδ)) +o(δ).

Rearrange and divide both sides byδ to get Gj(x)−Gj(x−Rjδ) δ = n X i=1 qijGi(x−Riδ) + X i∈Ω− αij Gi(0)−Gi(−Riδ) δ +o(δ). Letting δ→0, we get G0_j(x)Rj = n X i=1 Gi(x)·qij + X i∈Ω− αijRiG0i(0).

This shows thatG(x) is differentiable over (0, q). Later we shall show thatG0_j(0) = 0 if j ∈ Ω+ , (boundary condition (2.15c)). Hence we get Equation (2.14a), with

βj = n

P

i=1

G0_i(0)Riαij.

in the {X(t), t≥0}process from 0 to q. Thus Gj(x) = P{X(δ)> x, Z(δ) =j} = n P i=1 P{X(δ)> x, Z(δ) = j|X(0) > x−Riδ, Z(0) =i} ·P{X(0)> x−Riδ, Z(0) =i}.

Following the same steps as in x < q case, we get Equation (2.14b). This also proves that G(x) is differentiable forx∈(q,∞).

As for boundary conditions, Equation (2.15a) follows because G(x) is the com-plementary distribution function of the fluid level in steady state. The boundary condition (2.15b) for all states j /∈Ω0 is obvious from the fact that there is no

prob-ability mass at (q, j), i.e., Gj(x) is continuous at x =q, if j /∈Ω0. Equation (2.15c)

holds because 1/(G0_j(0)Rj) can be seen to be the expected time between two

consec-utive visits by the {(X(t), Z(t)), t ≥ 0} process to the state (0, j). If j ∈ Ω+, this

mean time is infinity. Hence G0_j(0) = 0 when j ∈ Ω+. From the definition of Gj(x)

we have Gj(0) = lim t→∞P{X(t)∈[0,∞), Z(t) = j}. Therefore, n P j=1 Gj(0) = 1, which is Equation (2.15d).

2.5

Solution to the Differential Equations

In this section, we give the solution to the differential equations (2.14a) and (2.14b). We shall treat the cases with distinct eigenvalues and repeated eigenvalues separately.

2.5.1

Case of Distinct Eigenvalues

Assume that all eigenvalues are distinct and hence the eigenvectors φi’s are linearly

independent. The next theorem gives the main result.

Theorem 2.4. The solution to the differential equations in Theorem 2.3 is given by

G(x) = m X i=1 cieλixφi+c0xπ+φ0, 0≤x≤q, (2.16) G(x) = m+ X i=1 aieλixφi, x > q, (2.17)

where the coefficients a1, a2, ..., am+, c1, c2, ..., cm, c0 and the vector φ0 are given by

the unique solution to the following system of linear equations:

m X i=1 ciφiQA+c0πR(A−I) +φ0Q= 0, (2.18a) m+ X i=1 aieλiqφij − m X i=1 cieλiqφij −c0qπj −φ0j = 0, j /∈Ω0, (2.18b) m X i=1 ciλiφij +c0πj = 0, j ∈Ω+, (2.18c) m X i=1 ciφi+φ0 ! e= 1, (2.18d) φ0 e= 1, (2.18e)

where φij is the j-th element in φi, i= 0,1, ..., m.

Proof. In section 2.2 we see that the homogenous equations (2.14b) have solutions of form G(x) = m X i=1 cieλixφi.

It can be shown that the nonhomogeneous equations (2.14a) have solutions of form G(x) = m X i=1 cieλixφi+c0xπ+φ0

if and only ifc0xπ+φ0 is a particular solution to (2.14a). UsingG(x) = c0xπ+φ0 in

(2.14a), we get

c0πR = c0xπQ+φ0Q+β

= φ0Q+β.

The last equation holds because πQ= 0. Substituting

β =G0(0)RA= m X i=1 ciλiφi +c0π ! RA

and noting that λiφiR=φiQ, we obtain

c0πR=φ0Q+

m

X

i=1

ciφiQA+c0πRA,

which can be rearranged to get Equation (2.18a). When x > q, G(x) has a solution of the form G(x) =

m+

P

i=1

aieλixφi (Note that boundary condition 2.15a implies that the

coefficient ai has to be zero when Re(λi)≥ 0). Because there is no probability mass

in (q, j) forj /∈Ω0 , the boundary condition in Equation (2.15b) reduces to

m+ X i=1 aieλiqφij = m X i=1 cieλiqφij +c0qπj +φ0j, j /∈Ω0, (2.19)

Rearranging (2.19) we get (2.18b). Equation (2.18c) and (2.18d) follow directly from boundary conditions (2.15c) and (2.15d).

The total number of unknown coefficients is m+ +m+n+ 1. Notice that the

number of independent equations in (2.18a) is n−1, since the rank of the matrix Q is n−1; the number of independent equations is m in (2.18b), and m+ in (2.18c).

Including Equation (2.18d) we have m++m+n independent equations satisfied by

m++m+n+ 1 coefficients. Since any particular solution will work, we use Equation

(2.18e) to determine a unique particular solution. Thus we have as many equations as unknowns.

2.5.2

Case of Repeated Eigenvalues

When there are repeated eigenvalues we solve this problem using “generalized” eigen-vectors. Let (λ1, φ (1) 1 ), (λ2, φ (2) 1 ), · · ·, (λK, φ (K)

1 ) be K solutions to Equation (2.5), and

λ1, λ2, ...λK are K distinct eigenvalues. Assume λ1, ...λK+ have negative real part,

λK++1 = 0, and λK++2, ...λK have positive real part. Let ni be the multiplicity of the

eigenvalue λi. Clearly ni ≥1 and K+ P i=1 ni =m+, K P i=K++2 ni =m−−1.

The general solution to the homogeneous equations G0(x)R=G(x)Q is given by

G(x) = K X i=1 eλix ni X j=1 c(_ji) j X k=1 xj−k (j−k)!φ (i) k , (2.20)

wherec(_ji)’s are constant coefficients, andφ(_ki)’s are “generalized” eigenvectors satisfy-ing φ(_ki)Q=λiφ (i) k R+φ (i) k−1R, k = 2, ..., ni.

Theorem 2.5. The solution to the differential equations in Theorem 2.3 is given by

G(x) = K X i=1 eλix ni X j=1 c(_ji) j X k=1 xj−k (j−k)!φ (i) k +c0xπ+φ0, if 0≤x≤q, G(x) = K+ X i=1 eλix ni X j=1 a(_ji) j X k=1 xj−k (j −k)!φ (i) k , if x > q,

where the coefficientsa(_ji)’s, c_j(i)’s,c0 and the vectorφ0 are given by the unique solution

to the following system of linear equations:

K X i=1 λi ni X j=1 c(_ji)φ(_ji)+ ni X j=2 c(_ji)φ(_ji₋)₁ ! RA+c0πR(A−I) +φ0Q= 0, K+ X i=1 eλiq ni X j=1 a(_ji) j X k=1 qj−k (j−k)!φ (i) kl − K X i=1 eλiq ni X j=1 c(_ji) j X k=1 qj−k (j−k)!φ (i) kl −c0qπl−φ0l = 0, l /∈Ω0, K X i=1 λi ni X j=1 c(_ji)φ(_jli)+ ni X j=2 c(_ji)φ(_ji₋)_1,l ! +c0πl = 0, l ∈Ω+, K X i=1 ni X j=1 c(_ji)φ(_ji)+φ0 ! e= 1, φ0 e= 1,

where φ(_kli) is the l-th element in φ(_ki).

Proof. Follow the same lines as in the proof of Theorem 2.4 with only changes in the general solution to the homogeneous equations.

Remark. When dealing with large matrices, the generalized eigenvectors are of-ten numerically difficult to compute. There are alternative methods that are numer-ically better behaved to evaluate the general solution to the homogeneous equations, e.g., Putzer (1966).

2.6

A Special Case:

A

=

I

In this section we consider a special case where the background process state does not change when the fluid level jumps to q, i.e., the case A=I.

2.6.1

Stochastic Decomposition Property

When A= I, as mentioned before, there exists an interesting stochastic decomposi-tion property of the limiting distribudecomposi-tion of the {(X(t), Z(t)), t ≥ 0} process, which says in steady state the buffer content in the fluid model with jumps is the sum of two independent random variables: a U(0, q) random variable and the buffer content in a fluid model with no jumps. Next we shall prove this decomposition property from two different aspects.

2.6.1.1 Laplace Stieltjes Transform Method

Let Fj(x) = lim t→∞P{X(t)≤x, Z(t) = j}, x≥0, j ∈Ω, (2.21) and ˜ Fj(s) = Z ∞ 0 e−sxdFj(x) (2.22)

be the Laplace Stieltjes transform (LST) of Fj(x), and

˜

F(s) = hF˜1(s),F˜2(s), ...,F˜n(s)

i

. (2.23)

The next theorem gives the stochastic decomposition property of the limiting distri-bution of the {(X(t), Z(t)), t≥0}process.

Theorem 2.6. Suppose A=I and the stability condition (2.10) holds. Then

˜

F(s) = 1−e −sq

sq H(s),˜ (2.24)

whereH(s)˜ is the LST of the limiting distribution function of the standard fluid model without jumps, given by Equation (2.9).

Proof. Since A = I and the stability condition (2.10) holds, we have G(0) = π and hence G(x) =π−F(x). Clearly, F(0) = 0. Thus

F0(x)R =F(x)Q+F0(0)R, 0≤x≤qi, F0(x)R =F(x)Q, x > qi. Thus we have Z q 0 e−sxF0(x)dx R= Z q 0 e−sx(F(x)Q+F0(0)R)dx and Z ∞ q e−sxF0(x)dx R = Z ∞ q e−sxF(x)Qdx . Thus ˜ F(s)R= Z ∞ 0 e−sxF(x)Qdx+ Z q 0 e−sxF0(0)Rdx = 1 s ˜ F(s)Q +1 s 1−e −sq F0(0)R. (2.25)

Rearranging Equation (2.25), we get

˜ F(s) = 1−e −sq sq sqF 0 (0)R(sR−Q)−1. (2.26)

Recalling from section 2.2, for the standard fluid model without jumps, we have

˜

H(s) =sH(0)R(sR−Q)−1. (2.27)

We have seen that there is a unique vector H(0) satisfying Equations (2.4a) and (2.4b) that makes ˜H(s) in Equation (2.27) a valid LST of a random vector. From

equation (2.26) it is clear that sqF0(0)R(sR−q)−1 _{must be a valid LST of a random}

vector since (1−e−sq_{)/(sq) is the LST ofU(0,1) random variable. Since the boundary}

conditions of Equation (2.15c) implies that F_j0(0) = 0 ifj ∈Ω+ , which are the same

conditions satisfied by H(0) (see Equation (2.4a)), we must have

qF0(0)R=cH(0)R,

for some constant c. The condition F(∞)e = 1 implies that c= 1. This proves our result.

Remark. Theorem 2.6 indicates that in steady state the buffer content in the fluid model with jumps is the sum of two independent random variables: a U(0, q) random variable and the buffer content in a fluid model with no jumps. Interestingly, similar property has been observed in queuing models with server vacations. See Fuhrmann (1984), Fuhrmann (1985) and Shanthikumar (1986).

2.6.1.2 Sample Path Method

We begin by decomposing the{X(t), t≥0}process into two components. LetS0 = 0,

X(0) =q and Si be the i-th order point (i≥1). Define

X1(t) = min

Sn≤u≤t

{X(u)}, Sn≤t < Sn+1

and

X2(t) =X(t)−X1(t).

Figure 2.2 illustrates the sample paths of the original {X(t), t ≥ 0} process and the two resulting processes {X1(t), t ≥ 0} and {X2(t), t ≥ 0}. The following two

Figure 2.2: Decomposition of the X(t) process. component processes {X1(t), t≥0} and {X2(t), t≥0}.

Theorem 2.7. The process {X2(t), t≥0} is independent of q.

Proof. Assume that Z(0) ∈Ω− and define

T1 = min{t≥0 :Z(t)∈Ω+∪Ω0}.

Regardless of the value of q, X(t) always decreases over (0, T1), except for possible

jumps of size q when it hits zero. Thus X2(t) is zero over (0, T1). T1 is independent

Now define

T2 = min{t > T1 :X(t) =X(T1)}.

Note that T2 is also independent of q, X2(T1) = X2(T2) = 0 and X2(t) > 0 for

t ∈ (T1, T2). The sample path of {X(t), t ∈(T1, T2)} is independent of q, since X(t)

never reaches 0 for any t∈(T1, T2). Thus the sample path of {X2(t), t ∈(T1, T2)} is

independent ofq. Define

T2n+1 = min{t ≥T2n :Z(t)∈Ω+∪Ω0},

and

T2n+2 = min{t≥T2n+1 :X(t) =X(T2n+1)}.

Since {X2(t), t ≥ 0} goes through these two cycles alternately over (T2n, T2n+1) and

(T2n+1, T2n+2) independently, it is clear that {X2(t), t≥0} is independent of q.

Theorem 2.8. The limiting distribution of the process{X1(t), t≥0}is uniform over

(0, q).

Proof. First note that the sample paths of {X1(t), t ≥ 0} have right derivative

everywhere. Define I(t) = 0 if the right derivative of X1(t) is strictly negative at t,

and I(t) = 1 if the right derivative of X1(t) is zero at t. Now

lim

t→∞P(X1(t)≤x) = lim

t→∞P(X1(t)≤x|I(t) = 0)P(I(t) = 0) + limt→∞P(X1(t)≤x|I(t) = 1)P(I(t) = 1). (2.28) Next we will show that

lim

First we construct two new processes{Y0(t), t≥0}and {Z0(t), t≥0}by eliminating

the segments of the sample paths of {X1(t), t ≥ 0} and {Z(t), t ≥ 0} over the time

intervals (T2n+1, T2n+2] for all n ≥ 0. The sample paths of the {Y0(t), t ≥ 0} and

{Z0(t), t ≥ 0} processes corresponding to the sample paths of {X1(t), t ≥ 0} and

{Z(t), t≥0} are shown in Figure 2.3. From Figure 2.3 we can see that{Y0(t), t≥0}

can be thought of as a fluid model modulated by the stochastic process{Z0(t), t≥0}

with state space Ω−. It can be seen that {Z0(t), t ≥ 0} is a CTMC with generator

matrix ˆQ= [ˆqij], (i, j ∈Ω−) given by

ˆ qij =qij + X k∈Ω+∪Ω0 qikηkj, i, j ∈Ω−, (2.30) where ηkj =P(Z(T2n+2) =j|Z(T2n+1) = k), k ∈Ω+∪Ω0, j ∈Ω−. (2.31)

Thus the{(Y0(t), Z0(t)), t≥0}process satisfies the hypothesis of Theorem 2.9. Hence

it follows that

lim

t→∞P(Y0(t)≤x, Z0(t) = i) = x

qπˆi, (2.32)

where ˆπi is the steady-state probability of the CTMC with generator matrix ˆQ in

state i. However, our construction of the Y0 process implies that

lim

t→∞P(Y0(t)≤x, Z0(t) = i) = limt →∞P(X1(t)≤x|I(t) = 0). This proves Equation (2.29) for ζ = 0.

Now for ζ = 1, we define Y1,n = X1(T2+n+1) and Z1,n = Z(T2+n+1) for n ≥ 0.

Now construct a semi-Markov process (SMP) {(Z1(t), Y1(t)), t ≥ 0} with embedded

T2n+2−T2n+1. Clearly the sample path of{Y1(t), t≥0}is identical to the one obtained

by eliminating the segments of the sample path of {X1(t), t ≥ 0} over the intervals

(T2n, T2n+1] for all n≥0. Figure 2.4 illustrates the sample paths of the {Y1(t), t≥0}

and {Z1(t), t≥0}processes corresponding to the sample paths of {X1(t), t ≥0} and

{Z(t), t≥0} processes. Define

f(j, x)dx= lim

t→∞P{Z(t) =j, x≤Y1(t)≤x+dx}. (2.33) According to the theory of SMP (see Kulkarni (1995)),

f(j, x)dx = _P π(j, x)τ(j, x)dx k∈Ω+∪Ω0 Rq y=0π(k, y)τ(k, y)dy , (2.34) where π(j, x)dx= lim n→∞P{Z1,n =j, x ≤Y1,n ≤x+dx}, (2.35) and τ(j, x) is the expected sojourn time of the SMP in state (j, x). Clearlyτ(j, x) is independent ofx, hence we denote τ(j, x) as τj for all x. Let

g(i, x)dx= lim

t→∞P{Z(t) =i, x≤Y0(t)≤x+dx}, (i∈Ω−). (2.36) From Equation (2.32), we see that

g(i, x) = πˆi

q , (i∈Ω−). (2.37)

Hence using Equation (2.37),

π(j, x) = X i∈Ω− g(i, x)qij = 1 q X i∈Ω− ˆ πiqij. (2.38)

Substituting Equation (2.38) into (2.34), we have f(j, x) = 1 q P i∈Ω− ˆ πiqijτj P k∈Ω+∪Ω0 Rq y=0 1 q P i∈Ω− ˆ πiqikτkdy = 1 q · P i∈Ω− ˆ πiqijτj P k∈Ω+∪Ω0 P i∈Ω− ˆ πiqikτk .

Thus the limiting probability density function of {Y1(t), t ≥0} process is given by

f(x) = X j∈Ω+∪Ω0 f(j, x) = 1 q · P j∈Ω+∪Ω0 P i∈Ω− ˆ πqˆijτj P k∈Ω+∪Ω0 P i∈Ω− ˆ πqˆikτk = 1 q. (2.39)

Equation (2.39) indicates the limiting distribution of {Y1(t), t ≥ 0} is uniform over

[0, q]. This proves Equation (2.29) for ζ = 1. Hence from (2.28)

lim

t→∞P(X1(t)≤x) = x q.

This proves theorem 2.8.

Remark. Using these two theorems we see that the limiting distribution of X(t) is a sum of two independent random variables: X1(t) is uniform and X2(t)

2.6.2

Uniform Limiting Distribution

Now we study a more specific case where Ri <0 for all i ∈ Ω, and the background

process state does not change when the buffer content jumps, i.e., A = I. Without loss of generality assume that X(0) = q. Then it is clear that X(t) ∈ [0, q], ∀t ≥ 0. The next theorem gives the steady state distribution of X(t).

Theorem 2.9. When R <0 and A=I,

G(x) = (1− 1

qx)π, x∈[0, q]. (2.40)

Proof. In this special case, the differential equations are reduced to

G0(x)R=G(x)Q+β, (2.41)

where

β =G0(0)R, (2.42)

with boundary conditions:

G(q) = 0, (2.43a)

G(0)e = 1. (2.43b)

It is easy to see that (2.40) is the solution to the differential equation system (2.41) with boundary conditions (2.43a) and (2.43b).

Remark. Theorem 2.9 implies that in steady state, the buffer content is uniformly distributed on [0, q], and is independent of the state of the environment. This is consistent with Theorem 2.6 since in this case the buffer content in the fluid model without jumps is zero with probability one in steady state. Similar results have been

observed by Browne and Zipkin (1991).

2.7

Examples

2.7.1

A Two-State Example

Consider a machine shop with only one machine. Whenever the machine is up, it produces items continuously at rater, and it fails after an exp(µ) amount of time. If it is down, there is no production, and it takes exp(λ) amount of time to fix it. The machine is as good as new after repairs complete. The demand occurs at a constant rated independent of the state of the machine. Whenever the inventory reaches zero, an external supply of amount q is ordered and arrives instantaneously.

This produces a special case of the model in Section 2.3 with the following pa-rameters: Q=    −λ λ µ −µ   , R =    −d 0 0 r−d   .

The stability condition Equation (2.10) reduces to λ(r−d)−µd < 0. We consider two cases.

Case 1: r > d. In this case, when the machine is up the production rate is greater than the demand rate. Thus the inventory hits zero only when the machine is down. We give explicit expressions of the limiting distribution in two sub-cases.

from the external supplier. The solution given in Theorem 2.4 reduces to Gdown(x) =      (r−d)π2 qdθ (e θx_−eθ(x−q)_{) x > q,} (r−d)π2 qdθ e θx₋ π1 qx+ π2 q(λ+µ)(d−r− rµ dθ) +π1 0≤x≤q, Gup(x) =      π2 qθ(e θx_−eθ(x−q)_{) x > q,} π2 qθe θx₋ π2 qx− π2 q(λ+µ)(d−r+ rλ dθ) +π2 0≤x≤q, where θ = λ(d_d−_(dr₋)+_r)dµ, π1 = _λ+µ_µ, and π2 = _λ+λ_µ. (2) A=    0 1 0 1  

.This implies that we replace the machine instantaneously if it is down when we place an order from the external supplier. In this case the solution is given by Gdown(x) =      λ(r−d)eθx+dµeθ(x−q) dθ(r+q(λ+µ)) when x > q, 1 r+q(λ+µ) _λ(r−d) θd e θx_+µx− rπ2µ θd −π1d +π1 when 0< x < q, Gup(x) =      (r−d)λeθx_−dµeθ(x−q) (r−d)θ(r+q(λ+µ)) when x > q, 1 r+q(λ+µ) λ θe θx_+λx− rπ2λ θd +π1d +π2 when 0< x < q.

Case 2: r < d. In this case the inventory can hit zero when the machine is either up or down.

given by Gdown(x) = 1−1 qx µ λ+µ, Gup(x) = 1−1 qx λ λ+µ. (2) A=    0 1 0 1  

. The solution on [0, q] is given by

Gdown(x) =

(d−r)dµeθx+µ(−rλ+d(λ+µ))eθqx+µeθq(dqµ−(d−r)(d−qλ))

−drµ−eθq_{(qrλ(λ+µ)−d(rµ+q(λ+µ)}2₎₎ ,

Gup(x) =

−d2_µeθx_{+λ(−rλ+d(λ+µ))e}θq_x+eθq_(q(d−r)λ2_+dµ(d+qλ))

−drµ−eθq_{(qrλ(λ+µ)−d(rµ+q(λ+µ)}2₎₎ .

Now we use λ= 1, µ= 2, d= 1, q= 1. We display the steady-state complementary cumulative distribution function Gup(x) +Gdown(x) and the density functionf(x) =

−G0_up(x)−G0_down(x) in Figure 2.5 and Figure 2.6 for the two cases when r= 0.5< d and r = 2.5> d. Note that the density functions in Figure 2.5 are discontinuous at x=qin this case. However the complementary cumulative distribution functions are always continuous.

Figure 2.6: Limiting distribution when r < d.

2.7.2

A Machine Shop Example

Now we consider a machine shop that hasnindependent and identical machines, each behaving as described in section 2.7.1. Each machine has its own repair person. Let Z(t) be the number of working machines at time t. Thus the environment process

{Z(t), t ≥ 0} has n+ 1 states, i.e., Ω = {0,1, ..., n}. Suppose the demand rate is directly proportional to the number of machines. To be specific, we have demand rate di = n and production rate ri = i·u for all i ∈ Ω, where u is the production

rate of each working machine. The background state does not change when placing an order. We plot the steady-state complementary cumulative distribution functions (ccdf) and the probability density functions (pdf) whenn = 3,d= 3, u= 2.5,λ = 1, µ= 2 andq= 3.

Figure 2.7: The steady-state ccdf.

Chapter 3

A Basic Production-Inventory

Model

3.1

Introduction

Beginning from this chapter we shall study a type of production-inventory models that can be seen as the fluid model in Chapter 2. We consider a single product problem. The production and demand rates are piecewise constant functions determined by an underlying exogenous CTMC. When the production rate exceeds the demand rate, the inventory increases, and when the demand rate exceeds the production rate, it decreases. Thus the inventory under continuous review is a fluid process that fluctuates according to the evolution of the underlying background process. This characterizes the situations in which the external environment undergoes recurring changes in a stochastic fashion, and can be regarded as Markovian.

We follow the classical reorder-point/order-quantity policy ((r,q) policy): when

r, a replenishment order of sizeq is placed from an external supplier. There are costs to hold products in inventory, to purchase and to produce. There is also a fixed set-up cost every time an order is placed with an external supplier. Our objective is to find the optimal (r, q) pair that minimizes the long-run average cost.

In the literature, in a deterministic setting with constant demand rate, the classical

Economic-Order-Quantity (EOQ) model describes the trade-off between the constant

set-up cost and the variable holding cost (see Zipkin (2000)). In such a model the demand occurs continuously at a constant rate d and there is a holding cost h per item per unit time. When the inventory reaches zero, an order of size qis placed and it arrives immediately. It costs k to place an order. The optimal value of q (EOQ) that minimizes the holding plus ordering cost per unit of time is given by

q∗ = r

2kd

h . (3.1)

We establish the stochastic EOQ theorem that shows in a CTMC environment the standard deterministic EOQ formula remains optimal if we replace deterministic de-mand rate by the expected net dede-mand rate in steady state.

In this chapter we first study a basic EOQ type model: no backlogging, and lead-time is zero. Section 3.2 models this inventory problem as a fluid-flow system. In Section 3.3 we prove the optimality of what we called stochastic version of the EOQ formula, which is derived by replacing the deterministic demand rate in the classical EOQ formula with the expected net demand rate in steady state. In Section 3.3.2 we calculate the minimum cost under that optimal ordering policy, and obtain the optimal production rate which is the best combination between outsourcing and pro-duction. Section 3.4 extends the basic model to allowing backlogging. We derive the optimal( reorder-point/order-quantity ) policy to achieve the trade-off point of fixed ordering cost, holding cost and backlogging cost under this stochastic circumstance.

For a given order quantity we show that the optimal reorder-point is given by the well-known newsboy solution. Particularly, in the special case where production is always less than the demand rate, we prove that the optimality of the deterministic EOQ formula with backlogging still holds in this stochastic environment.

3.2

The Model

We study a production-inventory system in which the inventory level process{X(t), t ≥

0}is modulated by a background process{Z(t), t≥0}. We assume that{Z(t), t≥0}

is an irreducible CTMC on state space Ω = {1,2, ..., n} with rate matrix Q = [qij].

When Z(t) is in state i, the production occurs continuously at a constant rate ri,

and demand occurs at rate di. As long as Z(t) = i, {X(t), t ≥ 0} changes at rate

Ri =ri−di. When the inventory level reaches the order point r, an order is placed

for the fixed amount q, the order quantity. Currently we assume leadtimes are zero, and no backlogging is allowed. This implies that it is optimal to set r = 0. We also assume all orders are of the same size q regardless of the state of the CTMC when the order is placed. This is an appropriate model when we can base our inventory re-plenishment decisions only on the inventory level and not on the state of the CTMC. This may be because knowledge of the state of the background CTMC is unavailable, or to simplify the ordering policies. (We will consider state dependent order sizes in Chapter 4 .) When the inventory level reaches 0, it jumps instantaneously to q, but there is no change in the state of the CTMC at this jump epoch.

Thus the {(X(t), Z(t))t ≥0} process can be seen to be a special case of the one studied in Chapter 2 withA=I. The stability condition is given by Equation (2.10).

Since A=I we know thatπ is the limiting distribution of Z(t). Thus ∆ =− n X i=1 πiRi (3.2)

is the net expected demand rate. The condition of stability can be written as ∆>0. This makes intuitive sense, since if ∆≤0, there would be no reason for placing orders from an external supplier.

3.3

Optimal Order Quantity

Next we consider the costs of operating the system. The total cost consists of three parts: holding cost, ordering cost, and production cost. We use the following notation:

h: cost to hold one item in inventory for one unit of time; k: fixed set-up cost whenever an order is placed;

p1: cost to purchase one item from the external supplier;

p2: cost to produce one item.

Let ch(q), co(q) and cp(q) be the steady-state holding, ordering and production

cost rates respectively as functions of the order quantity q. The total cost rate c(q) is given by

c(q) =ch(q) +co(q) +cp(q). (3.3)

3.3.1

Stochastic EOQ Theorem

Theorem 3.1. Suppose ∆> 0. Then the optimal order quantity q∗ that minimizes the total cost rate c(q) is given by

q∗ = r

2k∆

h . (3.4)

Proof. First we calculate ch(q). Recall in Section 2.6.1 we have shown that

when A = I, the limiting distribution of X(t) is a sum of two independent random variables: one is uniform (X1) and the other is the steady-state buffer content in a

standard fluid model without jumps (X2). ThusE(X1) = q₂ andE(X2) is independent

of q. Thus

ch(q) = hE(X)

=h(E(X1) +E(X2))

= hq

2 +hE(X2).

Next we calculate co(q). From the results on renewal reward processes we get

co(q) =

k+p1q

E(Si−Si−1)

,

where Si is the ith order point (i≤1).

In steady state, the average net demand during a cycle time (Si, Si−1) has to be

equal to the amount of the external supply. Hence we have

Thus co(q) = (k+p1q)∆ q = k∆ q +p1∆. (3.5) Finally cp(q) = p2 n X i=1 πiri, (3.6) and it is independent of q.

Now, from Equation (3.3), the total cost rate is given by

c(q) =ch(q) +co(q) +cp(q)

= hq 2 +

k∆ q +C,

whereC =hE(X2)+p1∆+cp(q) is independent ofq. Clearly,c(q) is a convex function

of q, and it is minimized at

q∗ = r

2k∆

h .

Remark. The optimal order quantity q∗ of Equation (3.4) is the classical EOQ formula with the deterministic demand rate replaced by the steady-state expected net demand rate.

A Numerical Example. Consider a machine shop as described in section 2.7.2. We investigate the effect of the production rate increases on the optimal order quantity q∗. Consider a system with λ = 1, µ= 2, h = 5, k = 0.5, p1 = 8 and p2 = 5. Let u

Figure 3.1: The optimal order quantity vs. production rate.

that for a fixed n, the q∗ decreases with u. This makes intuitive sense because as the production increases the net demand rate decreases. Note that the q∗ reaches zero whenu increases to 3. This is because the system is unstable foru≥3 and hence we do not need to order from the external supplier.

It should be noted that there are numerical difficulties when the parameter values make the rate Ri in some states close to zero. We have simply avoided such values

and used interpolation to produce the above graphs. This example is used repeatedly in this thesis, and this comment applies to all numerical experiments relating to it.

3.3.2

Minimum Cost Rate

Given the optimal order quantity q∗, we are able to calculate the corresponding min-imum cost rate:

First we calculate holding cost rate: ch(q∗) =hE(X) =h Z ∞ x=0 G(x)dx·e.

Substituting the expression for G(x) given by Equation (2.16) and (2.17), we have

ch(q∗) =h Z q∗ x=0 m X i=1 cieλixφi+c0xπ+φ0 ! dx+ Z ∞ x=q∗ m+ X i=1 aieλixφi ! dx ! e.

Recall that we have assumed that the eigenvaluesλ1, ..., λm+ have negative real parts,

λm++1 = 0, and λm++2, ..., λm have positive real parts. Thus

ch(q∗) =h   X i6=m++1 ci λi eλiq∗_−1φ i+ c0q∗ 2 2 π+q ∗ (φ0+cm++1φm++1)− m+ X i=1 ai λi eλix_φ i  ·e

Ordering cost rate and production cost rate can be obtained directly from Equation (3.5) and (3.6). Thus co(q∗) = k∆ q∗ +p1∆ cp(q∗) =p2 n X i=1 πiri,

A Numerical Example. Consider the machine shop example of Section 3.3.1. Given the optimal order quantity q∗, we calculate the corresponding minimum cost rate.

Figure 3.2: The minimum total cost and the optimal production rate.

From Figure 3.2 we can see that under the optimal ordering policy, the corresponding minimum cost rate is also a convex function of the production rateu. Thus there exists one optimal production rate u∗ that achieves the trade-off point between outsourcing and producing. Interestingly, even if the production cost rate p2 is less than the

outsourcing cost ratep1, the optimal policy does not suggest us depending on

inhouse-production too much, in which circumstance the increase in holding cost due to the high production rate annihilates the advantage of the cost difference between the purchasing inhouse-production.

Also note that althoughq∗ →0 asu↑3, the expected total cost does not approach zero. This is a consequence of the stochastic model, as opposed to the deterministic one.

3.4

Inventory Model with Backlogging

In the previous sections we considered a model where we place an order as soon as the inventory on hand is zero. Many businesses find it practical to operate with planned backlogging. In this section we consider the same system as in the sections above, but allow backlogging, and assume that unsatisfied demands are fully backlogged. Let X(t) be the net inventory level at time t (i.e., the inventory on hand at time t - backorders at time t). We always use any inventory on hand to fill demands; backorders accumulate only when we run of stock entirely. Thus if X(t) is positive, it represents the amount of inventory on hand. If it is negative, it represents the negative of the amount of backorders at time t.

Now besides all the costs occurring in the previous setting, there is alsobacklogging cost. We consider a policy under which we place an order of size q whenever the inventory level decreases to the reorder point r. We assume zero leadtimes, so the orders arrive instantiates. Clearly an optimal policy should have r ≤ 0 to reduce unnecessary holding cost. However, unlike in the deterministic set up, we may not have q > −r. The net inventory level is always in (r,∞). Figure 3.3 illustrates a typical sample path of the{X(t), t ≥0} process.