Discrete-time stochastic processes

# Let i 1 i k be the solution for given 0 to j k x i0 i

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ; 0 ≤ i, j ≤ k. (6.63) Note that ∏i Qij represents the rate at which forward transitions go from i to j , and ∏i represents the rate at which forward transitions leave node i. Equation (6.63) takes advantage of the fact that ∏i is also the rate at which forward transitions enter node i, and thus the rate at which backward transitions leave node i. Using the conjecture that the backward time system is a Jackson network with routing probabilities {Q∗ ; 0 ≤ i, j ≤ k}, we can ij write down the backward transition rates in the same way as (6.59-6.61), ∗ qm ,m 0 = ∏0 Q∗j 0 = = for m 0 = m + e j µi Q∗0 i µi Q∗ ij (6.64) 0 (6.65) 0 (6.66) for m = m − e i , mi > 0, 1 ≤ i ≤ k for m = m − e i + e j , m − i > 0, 1 ≤ i, j ≤ k. If we substitute (6.63) into (6.64)-(6.66), we obtain ∗ qm ,m 0 = ∏j Qj 0 for m 0 = m + e j , = (µi /∏i )∏0 Q0i = (µi /∏i )∏j Qj i 0 1≤j≤k for m = m − e i , mi > 0, 0 for m = m − e i + e j , (6.67) 1≤i≤k (6.68) mi > 0, 1 ≤ i, j ≤ k. (6.69) This gives us our hypothesized backward transition rates in terms of the parameters of the original Jackson network. To use theorem 6.5, P must verify that there is a set of positive we P numbers, p(m ), satisfying m p(m ) = 1 and m ∫m pm < 1, and a set of non-negative ∗ numbers qm 0 ,m satisfying the following two sets of equations: ∗ p(m )qm ,m 0 = p(m 0 )qm 0 ,m X X ∗ qm ,m 0 = qm ,m 0 m for all m , m 0 (6.70) for all m . (6.71) m0 We verify (6.70) by substituting (6.59)-(6.61) on the left side of (6.70) and (6.67)-(6.69) on the right side. Recalling that ρi is deﬁned as ∏i /µi , and cancelling out common terms on each side, we have p(m ) = p(m 0 )/ρj 0 p(m ) = p(m )ρi 0 p(m ) = p(m )ρi /ρj for m 0 = m + e j (6.72) 0 for m = m − e i , mi > 0 0 for m = m − e i + e j , (6.73) mi > 0. (6.74) Looking at the case m 0 = m − e i , and using this equation repeatedly to get from state (0, 0, . . . , 0) up to an arbitrary m, we obtain p(m ) = p(0, 0, . . . , 0) k Y i=1 ρmi . i (6.75) 6.7. JACKSON NETWORKS 263 It is easy to verify that (6.75) satisﬁes (6.72)-(6.74) for all possible transitions. Summing over all m to solve for p(0, 0, . . . , 0), we get X X X Xm 1= p(m ) = p(0, 0, . . . , 0) ρm1 ρm2 . . . ρk k 1 2 m1 ,m2 ,... ,mk m1 m2 −1 = p(0, 0 . . . , 0)(1 − ρ1 ) mk −1 (1 − ρ2 ) . . . (1 − ρk )−1 . Thus, p(0, 0, . . . , 0) = (1 − ρ1 )(1 − ρ2 ) . . . (1 − ρk ), and substituting this in (6.75), we get p(m ) = k Y i=1 k i Yh pi (mi ) = (1 − ρi )ρmi . i (6.76) i=1 where pi (m) = (1 − ρi )ρm is the steady state distribution of a single M/M/1 queue. Now i that we have found the steady state distribution implied by our assumption about the backward process being a Jackson network, our remaining task is to verify (6.71) P P ∗ To verify (6.71), i.e., m 0 qm ,m 0 = m 0 qm ,m 0 , ﬁrst consider the right side. Using (6.64) to sum over all m 0 = m + e j , then (6.65) to sum over m 0 = m − e i (for i such that mi > 0), and ﬁnally (6.66) to sum over m 0 = m − e i + e j , (again for i such that mi > 0), we get X m0 ∗ qm ,m 0 = k X ∏0 Q∗j + 0 j =1 X µi Q∗0 + i i:mj >0 X i:mi >0 Using the fact Q∗ is a stochastic matrix, then, X X ∗ qm ,m 0 = ∏0 + µi . m0 µi k X Q∗ . ij (6.77) j =1 (6.78) i:mi >0 The left hand side of (6.71) can be summed in the same way to get the result on the right side of (6.78), but we can see that P must be the result by simply observing that ∏0 is this the rate of exogenous arrivals and i:mi>0 µi is P overall rate of P the service completions in state m . Note that this also veriﬁes that ∫m = m 0 qm ,m 0 ≥ ∏0 + i µi , and since ∫m is P bounded, m ∫m p(m ) < 1. Since all the conditions of Theorem 6.5 are satisﬁed, p(m ), as given in (6.76), gives the steady state probabilities for the Jackson network. This also veriﬁes that the backward process is a Jackson network, and hence the exogenous departures are Poisson and independent. Although the exogenous arrivals and departures in a Jackson network are Poisson, the endogenous processes of customers travelling from one node to another are typically not Poisson if there are feedback paths in the network. Also, although (6.76) shows that the numbers of customers at the diﬀerent nodes are independent random variables at any given time in steady state, it is not generally true that the number of customers at one node at one time is independent of the number of customers at another node at another time. There are many generalizations of the reversibility arguments used above, and many network situations in which the nodes have independent states at a common time. We discuss just two of them here and refer to Kelly, [13], for a complete treatment. 264 CHAPTER 6. MARKOV PROCESSES WITH COUNTABLE STATE SPACES For the ﬁrst generalization, assume that the service time at each node depends on the number of customers at that node, i.e., µi is replaced by µi,mi . Note that this includes the M/M/m type of situation in which each node has several independent exp...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online