Discrete-time stochastic processes

56 round robin and processor sharing typical queueing

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: e generation to the next. The individuals could be photons in a photomultiplier, particles in a cloud chamber, micro-organisms, insects, or branches in a data structure. Let Xn be the number of individuals in generation n of some population. Each of these Xn individuals, independently of each other, produces a random number of offspring, and these offspring collectively make up generation n + 1. More precisely, a branching process is a Markov chain in which the state Xn at time n models the number of individuals in generation n. Denote the individuals of generation n as {1, 2, ..., Xn } and let Yk,n be the number of offspring of individual k. The random variables Yk,n are defined to be IID over k and n, with a PMF pj = Pr {Yk,n = j }. The state at time n + 1, namely the number of individuals in generation n + 1, is Xn+1 = Xn X Yk,n . (5.21) k=1 Assume a given distribution (perhaps deterministic) for the initial state X0 . The transition probability, Pij = Pr {Xn+1 = j | Xn = i}, is just the probability that Y1,n +Y2,n +· · ·+Yi,n = 208 CHAPTER 5. COUNTABLE-STATE MARKOV CHAINS j . The zero state (i e., the state in which there are no individuals) is a trapping state (i.e., P00 = 1) since no future offspring can arise in this case. One of the most important issues about a branching process is the probability that the population dies out eventually. Naturally, if p0 (the probability that an individual has no offspring) is zero, then each generation must be at least as large as the generation before, and the population cannot die out unless X0 = 0. We assume in what follows that p0 > 0 and X0 > 0. Recall that Fij (n) was defined as the probability, given X0 = i, that state j is entered between times 1 and n. From (5.4), this satisfies the iterative relation X Fij (n) = Pij + Pik Fkj (n − 1), n > 1; Fij (1) = Pij . (5.22) k6=j The probability that the process dies out by time n or before, given X0 = i, is thus Fi0 (n). For the nth generation to die out, starting with an initial population of i individuals, the descendants of each of those i individuals must die out. Since each individual generates descendants independently, we have Fi0 (n) = [F10 (n)]i for all i and n. Because of this relationship, it is sufficient to find F10 (n), which can be determined from (5.22). Observe that P1k is just pk , the probability that an individual will have k offspring. Thus, (5.22) becomes F10 (n) = p0 + 1 X k=1 k pk [F10 (n − 1)] = 1 X k=0 pk [F10 (n − 1)]k . (5.23) P Let h(z ) = k pk z k be the z transform of the number of an individual’s offspring. Then (5.23) can be written as F10 (n) = h(F10 (n − 1)). (5.24) This iteration starts with F10 (1) = p0 . Figure 5.3 shows a graphical construction for evaluating F10 (n). Having found F10 (n) as an ordinate on the graph for a given value of n, we find the same value as an abscissa by drawing a horizontal line over to the straight line of slope 1; we then draw a vertical line back to the curve h(z ) to find h(F10 (n)) = F10 (n + 1). For the two figures shown, it can be seen that F10 (1) is equal to the smallest root of the equation h(z ) − z = 0. We next show that these two figures are representative of all possibilities. Since h(z ) is a z transform, we know P h(1) = 1, so that z = 1 is one that root of h(z ) − z = 0. Also, h0 (1) = Y , where Y = k kpk is the expected number of an individual’s offspring. If Y > 1, as in Figure 5.3a, then h(z ) − z is negative for z slightly smaller than 1. Also, for z = 0, h(z ) − z = h(0) = p0 > 0. Since h00 (z ) ≥ 0, there is exactly one root of h(z ) − z = 0 for 0 < z < 1, and that root is equal to F10 (1). By the same type of analysis, it can be seen that if Y ≤ 1, as in Figure 5.3b, then there is no root of h(z ) − z = 0 for z < 1, and F10 (1) = 1. As we saw earlier, Fi0 (1) = [F10 (1)]i , so that for any initial population size, there is a probability strictly between 0 and 1 that successive generations eventually die out for Y > 1, and probability 1 that successive generations eventually die out for Y ≤ 1. Since state 0 is accessible from all i, but F0i (1) = 0, it follows from Lemma 5.3 that all states other than state 0 are transient. 5.2. BRANCHING PROCESSES 1 209 ° ° ° h(z ) ° ° F10 (1) ° F10 (3) ° F10 (2) ° 10 (1) p0 F ° ° z 1 F10 (1) ° ° ° ° F (3) ° F (2) 10 h(z ) 10 ° p0 °F10 (1) ° ° ° Figure 5.3: Graphical construction to find the probability that a population dies out. Here F10 (n) is the probability that a population starting with one member at generation 0 dies out by generation n or before. Thus F10 (1) is the probability that the population ever dies out. We next evaluate the expected number of individuals in a given generation. Conditional on Xn−1 = i, (5.21) shows that the expected value of Xn is iY . Taking the expectation over Xn−1 , we have E [Xn ] = Y E [Xn−1 ] . (5.25) Iterating this equation, we get n E [Xn ] = Y E [X0 ] . (5.26) Thus, if Y > 1, the expected number of in...
View Full Document

This note was uploaded on 09/27/2010 for the course EE 229 taught by Professor R.srikant during the Spring '09 term at University of Illinois, Urbana Champaign.

Ask a homework question - tutors are online