Discrete-time stochastic processes

# Discrete-time stochastic processes

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ition rates in the backward process. Proof: Sum (6.57) over i. Using the fact that X pi qij = pj ∫j P j qij = ∫i and using (6.56), we obtain for allj. (6.58) i P These, along with i pi = 1, are the steady state equations for the process. These equations thus have a solution, and by Theorem 6.2, pi > 0 for all i, the embedded chain is positive ∗ recurrent, and pi = limt→1 Pr {X (t) = i}. Finally, qij as given by (6.57) is the backward transition rate as given by (6.54) for all i, j . ∗ We see that Theorem 6.4 is just a special case of Theorem 6.5 in which the guess about qij ∗ =q . is that qij ij Birth-death processes are all reversible if the steady state probabilities exist. To see this, note that Equation (6.40) (the equation to ﬁnd the steady state probabilities) is just (6.55) applied to the special case of birth-death processes. Due to the importance of this, we state it as a theorem. Theorem 6.6. For a birth-death process, if there is a solution {pi ; i ≥ 0} to (6.40) with P P i pi = 1 and i pi ∫i < 1, then the process is reversible, and the embedded chain is positive recurrent and reversible. Since the M/M/1 queueing process is a birth-death process, it is also reversible. Burke’s theorem, which was given as Theorem 5.8 for sampled-time M/M/1 queues, can now be established for continuous-time M/M/1 queues. Note that the theorem here contains an extra part, part c). Theorem 6.7 (Burke’s theorem). Given an M/M/1 queueing system in steady state with ∏ < µ, 256 CHAPTER 6. MARKOV PROCESSES WITH COUNTABLE STATE SPACES a) the departure process is Poisson with rate ∏, b) the state X (t) at any time t is independent of departures prior to t, and c) for FCFS service, given that a customer departs at time t, the arrival time of that customer is independent of the departures prior to t. Proof: The proofs of parts a) and b) are the same as the proof of Burke’s theorem for sampled-time, Theorem 5.8, and thus will not be repeated. For part c), note that with FCFS service, the mth customer to arrive at the system is also the mth customer to depart. Figure 6.9 illustrates that the association between arrivals and departures is the same in the backward system as in the forward system (even though the customer ordering is reversed in the backward system). In the forward, right moving system, let τ be the epoch of some given arrival. The customers arriving after τ wait behind the given arrival in the queue, and have no eﬀect on the given customer’s service. Thus the interval from τ to the given customer’s service completion is independent of arrivals after τ . r a1 d4 r r a2 r a3 r d3 a4 r Right moving (forward) M/M/1 process d3 r d1 r r a4 d2 r d2 r r a3 r a2 d1 r Left moving (backward) M/M/1 process d4 r r a1 Figure 6.9: FCFS arrivals and departures in right and left moving M/M/1 processes. Since the backward, left moving, system is also an M/M/1 queue, the interval from a given backward arrival, say at epoch t, moving left until the corresponding departure, is independent of arrivals to the left of t. From the correspondence between sample functions in the right moving and left moving systems, given a departure at epoch t in the right moving system, the departures before time t are independent of the arrival epoch of the given customer departing at t; this completes the proof. Part c) of Burke’s theorem does not apply to sampled-time M/M/1 queues because the sampled time model does not allow for both an arrival and departure in the same increment of time. Note that the proof of Burke’s theorem (including parts a and b from Section 5.5) does not make use of the fact that the transition rate qi,i−1 = µ for i ≥ 1 in the M/M/1 queue. Thus Burke’s theorem remains true for any birth death Markov process in steady state for which qi,i+1 = ∏ for all i ≥ 0. For example, parts a and b are valid for M/M/m queues; part c is also valid (see [Wol89]), but the argument here is not adequate since the ﬁrst customer to enter the system might not be the ﬁrst to depart. 6.6. REVERSIBILITY FOR MARKOV PROCESSES 257 We next show how Burke’s theorem can be used to analyze a tandem set of queues. As shown in Figure 6.10, we have an M/M/1 queueing system with Poisson arrivals at rate ∏ and service at rate µ1 . The departures from this queueing system are the arrivals to a second queueing system, and we assume that a departure from queue 1 at time t instantaneously enters queueing system 2 at the same time t. The second queueing system has a single server and the service times are IID and exponentially distributed with rate µ2 . The successive service times at system 2 are also independent of the arrivals to systems 1 and 2, and independent of the service times in system 1. Since we have already seen that the departures from the ﬁrst system are Poisson with rate ∏, the arrivals to the second queue are Poisson with rate ∏. Thus the second system is also M/M/1. M/M/1 ∏✲ M/M/1 ∏✲ µ1 ∏✲ µ2 Figure 6.10: A tandem queueing syst...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online