Discrete-time stochastic processes

# B find the time average fraction of time that the

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: howed that they caused no problems in our discussion of uniformization. It is convenient to allow these self transitions here, partly for the added generality and partly to illustrate that the single node network with feedback of Figure 6.11 is an example of a Jackson network. Our ob jective is to ﬁnd the steady state probabilities p(m ) for this type of process, and our plan of attack is in accordance with Theorem 6.5; that is, we shall guess a set of transition rates for the backward Markov process, use these to guess p(m ), and then verify that the guesses are correct. Before making these guesses, however, we must ﬁnd out a little more about how the system works, so as to guide the guesswork. Let us deﬁne ∏i for each i, 6.7. JACKSON NETWORKS 261 1 ≤ i ≤ k, as the time average overall rate of arrivals to node i, including both exogenous and endogenous arrivals. Since ∏0 is the rate of exogenous inputs, we can interpret ∏i /∏0 as the expected number of visits to node i per exogenous input. The endogenous arrivals to node i are not necessarily Poisson, as the example of a single queue with feedback shows, and we are not even sure at this point that such a time-average rate exists in any reasonable sense. However, let us assume for the time being that such rates exist and that the timeaverage rate of departures from each node equals the time-average rate of arrivals (i.e., the queue sizes do not grow linearly with time). Then these rates must satisfy the equation ∏j = k X j =0 ∏i Qij ; 1 ≤ j ≤ k. (6.62) To see this, note that ∏0 Q0j is the rate of exogenous arrivals to j . Also ∏i is the time-average rate at which customers depart from queue i, and ∏i Qij is the rate at which customers go from node i to node j . Thus, the right hand side of (6.62) is the sum of the exogenous and endogenous arrival rates to node j . Note the distinction between the time-average rate of customers going from i to j in (6.62) and the rate qm ,m 0 = µi Qij for m 0 = m −e i +e j , mi &gt; 0 in (6.61). The rate in (6.61) is conditioned on a state m with mi &gt; 0, whereas that in (6.62) is the overall time-average rate, averaged over all states. Note that {Qij ; 0 ≤ i, j ≤ k} forms a stochastic matrix and (6.62) is formally equivalent to the equations for steady state probabilities (except that steady state probabilities sum to 1). The usual equations for steady state probabilities include an equation for j = 0, but that equation is redundant. Thus we know that, if there is a path between each pair of nodes (including the ﬁctitious node 0), then (6.62) has a solution for {∏i ; 0 ≤ i ≤ k, and that solution is unique within a scale factor. The known value of ∏0 determines this scale factor and makes the solution unique. Note that we don’t have to be careful at this point about whether these rates are time averages in any nice sense, since this will be veriﬁed later; we do have to make sure that (6.62) has a solution, however, since it will appear in our solution for p(m ). Thus we assume in what follows that a path exists between each pair of nodes, and thus that (6.62) has a unique solution as a function of ∏0 . We now make the ﬁnal necessary assumption about the network, which is that µi &gt; ∏i for each node i. This will turn out to be required in order to make the process positive recurrent. We also deﬁne ρi as ∏i µi . We shall ﬁnd that, even though the inputs to an individual node i are not Poisson in general, there is a steady state distribution for the number of customers at i, and that distribution is the same as that of an M/M/1 queue with the parameter ρi . Now consider the backward time process. We have seen that only three kinds of transitions are possible in the forward process. First, there are transitions from m to m 0 = m + e j for any j , 1 ≤ j ≤ k. Second, there are transitions from m to m − e i for any i, 1 ≥ i ≤ k, such that mi &gt; 0. Third, there are transitions from m to m 0 = m − e i + e j for 1 ≤ i, j ≤ k with mi &gt; 0. Thus in the backward process, transitions from m 0 to m are possible only for the m , m 0 pairs above. Corresponding to each arrival in the forward process, there is a departure in the backward process; for each forward departure, there is a backward arrival; and for each forward passage from i to j , there is a backward passage from j to i. 262 CHAPTER 6. MARKOV PROCESSES WITH COUNTABLE STATE SPACES We now make the conjecture that the backward process is itself a Jackson network with Poisson exogenous arrivals at rates {∏0 Q∗j }, service times that are exponential with rates 0 {µi }, and routing probabilities {Q∗ }. The backward routing probabilities {Q∗ } must ij ij be chosen to be consistent with the transition rates in the forward process. Since each transition from i to j in the forward process must correspond to a transition from j to i in the backward process, we should have ∏i Qij = ∏j Q∗i j...
View Full Document

## This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online