Unformatted text preview: tion probabilities for
the backward chain. Proof: Summing (5.41) over i, we get the steadystate equations for the Markov chain, so
the fact that the given {πi } satisfy these equations asserts that they are the steadystate
∗
probabilities. Equation (5.41) then asserts that {Pij } is the set of transition probabilities
for the backward chain.
The following two sections illustrate some important applications of reversibility. 5.5 The M/M/1 sampletime Markov chain The M/M/1 Markov chain is a sampledtime model of the M/M/1 queueing system. Recall
that the M/M/1 queue has Poisson arrivals at some rate ∏ and IID exponentially distributed
service times at some rate µ. We assume throughout this section that ∏ < µ (this is
required to make the states positiverecurrent). For some given small increment of time 5.5. THE M/M/1 SAMPLETIME MARKOV CHAIN 215 δ , we visualize observing the state of the system at the sample times nδ . As indicated in
Figure 5.5, the probability of an arrival in the interval from (n − 1)δ to nδ is modeled as
∏δ , independent of the state of the chain at time (n − 1)δ and thus independent of all prior
arrivals and departures. Thus the arrival process, viewed as arrivals in subsequent intervals
of duration δ , is Bernoulli, thus approximating the Poisson arrivals. This is a sampledtime
approximation to the Poisson arrival process of rate ∏ for a continuous time M/M/1 queue.
♥
0
②
❖ ∏δ
µδ ③
♥
1
②
❖ ∏δ
µδ ③
♥
2
②
❖ ∏δ
µδ ③
♥
3
②
❖ ∏δ
µδ ③
♥
4 ... ❖ Figure 5.5: Sampledtime approximation to M/M/1 queue for time increment δ .
When the system is nonempty (i.e., the state of the chain is one or more), the probability
of a departure in the interval (n − 1)δ to nδ is µδ , thus modelling the exponential service
times. When the system is empty, of course, departures cannot occur.
Note that in our sampledtime model, there can be at most one arrival or departure in an
interval (n − 1)δ to nδ . As in the Poisson process, the probability of more than one arrival,
more than one departure, or both an arrival and a departure in an increment δ is of order
δ 2 for the actual continuous time M/M/1 system being modeled. Thus, for δ very small,
we expect the sampledtime model to be relatively good. At any rate, we can now analyze
the model with no further approximations.
Since this chain is a birthdeath chain, we can use (5.30) to determine the steadystate
probabilities; they are
πi = π0 ρi ; ρ = ∏/µ < 1.
Setting the sum of the πi to 1, we ﬁnd that π0 = 1 − ρ, so
πi = (1 − ρ)ρi ; all i ≥ 0. (5.42) Thus the steadystate probabilities exist and the chain is a birthdeath chain, so from
Theorem 5.5, it is reversible. We now exploit the consequences of reversibility to ﬁnd some
rather surprising results about the M/M/1 chain in steadystate. Figure 5.6 illustrates a
sample path of arrivals and departures for the chain. To avoid the confusion associated
with the backward chain evolving backward in time, we refer to the original chain as the
chain moving to the right and to the backward chain as the chain moving to the left.
There are two types of correspondence between the rightmoving and the leftmoving chain:
1. The leftmoving chain has the same Markov chain description as the rightmoving
chain, and thus can be viewed as an M/M/1 chain in its own right. We still label the
sampledtime intervals from left to right, however, so that the leftmoving chain makes
transitions from Xn+1 to Xn to Xn−1 . Thus, for example, if Xn = i and Xn−1 = i + 1,
the leftmoving chain has an arrival in the interval from nδ to (n − 1)δ . 216 CHAPTER 5. COUNTABLESTATE MARKOV CHAINS 2. Each sample function . . . xn−1 , xn , xn+1 . . . of the rightmoving chain corresponds to
the same sample function . . . xn+1 , xn , xn−1 . . . of the leftmoving chain, where Xn−1 =
xn−1 is to the left of Xn = xn for both chains. With this correspondence, an arrival
to the rightmoving chain in the interval (n − 1)δ to nδ is a departure from the leftmoving chain in the interval nδ to (n − 1)δ , and a departure from the rightmoving
chain is an arrival to the leftmoving chain. Using this correspondence, each event in
the leftmoving chain corresponds to some event in the rightmoving chain.
In each of the properties of the M/M/1 chain to be derived below, a property of the leftmoving chain is developed through correspondence 1 above, and then that property is
translated into a property of the rightmoving chain by correspondence 2.
Property 1: Since the arrival process of the rightmoving chain is Bernoulli, the arrival
process of the leftmoving chain is also Bernoulli (by correspondence 1). Looking at a
sample function xn+1 , xn , xn−1 of the leftmoving chain (i.e., using correspondence 2), an
arrival in the interval nδ to (n − 1)δ of the leftmoving chain is a departure in the interval
(n − 1)δ to nδ of the rightmoving chain. Since the arrivals in successive increments of the
le...
View
Full
Document
This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.
 Spring '09
 R.Srikant

Click to edit the document details