Discrete-time stochastic processes

# Since u i is the expected holding time in i per

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: tion probabilities for the backward chain. Proof: Summing (5.41) over i, we get the steady-state equations for the Markov chain, so the fact that the given {πi } satisfy these equations asserts that they are the steady-state ∗ probabilities. Equation (5.41) then asserts that {Pij } is the set of transition probabilities for the backward chain. The following two sections illustrate some important applications of reversibility. 5.5 The M/M/1 sample-time Markov chain The M/M/1 Markov chain is a sampled-time model of the M/M/1 queueing system. Recall that the M/M/1 queue has Poisson arrivals at some rate ∏ and IID exponentially distributed service times at some rate µ. We assume throughout this section that ∏ < µ (this is required to make the states positive-recurrent). For some given small increment of time 5.5. THE M/M/1 SAMPLE-TIME MARKOV CHAIN 215 δ , we visualize observing the state of the system at the sample times nδ . As indicated in Figure 5.5, the probability of an arrival in the interval from (n − 1)δ to nδ is modeled as ∏δ , independent of the state of the chain at time (n − 1)δ and thus independent of all prior arrivals and departures. Thus the arrival process, viewed as arrivals in subsequent intervals of duration δ , is Bernoulli, thus approximating the Poisson arrivals. This is a sampled-time approximation to the Poisson arrival process of rate ∏ for a continuous time M/M/1 queue. ♥ 0 ② ❖ ∏δ µδ ③ ♥ 1 ② ❖ ∏δ µδ ③ ♥ 2 ② ❖ ∏δ µδ ③ ♥ 3 ② ❖ ∏δ µδ ③ ♥ 4 ... ❖ Figure 5.5: Sampled-time approximation to M/M/1 queue for time increment δ . When the system is non-empty (i.e., the state of the chain is one or more), the probability of a departure in the interval (n − 1)δ to nδ is µδ , thus modelling the exponential service times. When the system is empty, of course, departures cannot occur. Note that in our sampled-time model, there can be at most one arrival or departure in an interval (n − 1)δ to nδ . As in the Poisson process, the probability of more than one arrival, more than one departure, or both an arrival and a departure in an increment δ is of order δ 2 for the actual continuous time M/M/1 system being modeled. Thus, for δ very small, we expect the sampled-time model to be relatively good. At any rate, we can now analyze the model with no further approximations. Since this chain is a birth-death chain, we can use (5.30) to determine the steady-state probabilities; they are πi = π0 ρi ; ρ = ∏/µ < 1. Setting the sum of the πi to 1, we ﬁnd that π0 = 1 − ρ, so πi = (1 − ρ)ρi ; all i ≥ 0. (5.42) Thus the steady-state probabilities exist and the chain is a birth-death chain, so from Theorem 5.5, it is reversible. We now exploit the consequences of reversibility to ﬁnd some rather surprising results about the M/M/1 chain in steady-state. Figure 5.6 illustrates a sample path of arrivals and departures for the chain. To avoid the confusion associated with the backward chain evolving backward in time, we refer to the original chain as the chain moving to the right and to the backward chain as the chain moving to the left. There are two types of correspondence between the right-moving and the left-moving chain: 1. The left-moving chain has the same Markov chain description as the right-moving chain, and thus can be viewed as an M/M/1 chain in its own right. We still label the sampled-time intervals from left to right, however, so that the left-moving chain makes transitions from Xn+1 to Xn to Xn−1 . Thus, for example, if Xn = i and Xn−1 = i + 1, the left-moving chain has an arrival in the interval from nδ to (n − 1)δ . 216 CHAPTER 5. COUNTABLE-STATE MARKOV CHAINS 2. Each sample function . . . xn−1 , xn , xn+1 . . . of the right-moving chain corresponds to the same sample function . . . xn+1 , xn , xn−1 . . . of the left-moving chain, where Xn−1 = xn−1 is to the left of Xn = xn for both chains. With this correspondence, an arrival to the right-moving chain in the interval (n − 1)δ to nδ is a departure from the leftmoving chain in the interval nδ to (n − 1)δ , and a departure from the right-moving chain is an arrival to the left-moving chain. Using this correspondence, each event in the left-moving chain corresponds to some event in the right-moving chain. In each of the properties of the M/M/1 chain to be derived below, a property of the leftmoving chain is developed through correspondence 1 above, and then that property is translated into a property of the right-moving chain by correspondence 2. Property 1: Since the arrival process of the right-moving chain is Bernoulli, the arrival process of the left-moving chain is also Bernoulli (by correspondence 1). Looking at a sample function xn+1 , xn , xn−1 of the left-moving chain (i.e., using correspondence 2), an arrival in the interval nδ to (n − 1)δ of the left-moving chain is a departure in the interval (n − 1)δ to nδ of the right-moving chain. Since the arrivals in successive increments of the le...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online