This preview shows page 1. Sign up to view the full content.
Unformatted text preview: dividuals in a generation increases exponentially
with n, and Y gives the rate of growth. Physical processes do not grow exponentially
forever, so branching processes are appropriate models of such physical processes only over
some ﬁnite range of population. Even more important, the model here assumes that the
number of oﬀspring of a single member is independent of the total population, which is
highly questionable in many areas of population growth. The advantage of an oversimpliﬁed
model such as this is that it explains what would happen under these idealized conditions,
thus providing insight into how the model should be changed for more realistic scenarios.
It is important to realize that, for branching processes, the mean number of individuals
is not a good measure of the actual number of individuals. For Y = 1 and X0 = 1, the
expected number of individuals in each generation is 1, but the probability that Xn = 0
approaches 1 with increasing n; this means that as n gets large, the nth generation contains
a large number of individuals with a very small probability and contains no individuals with
a very large probability. For Y > 1, we have just seen that there is a positive probability
that the population dies out, but the expected number is growing exponentially.
A surprising result, which is derived from the theory of martingales in Chapter 7, is that
n
if X0 = 1 and Y > 1, then the sequence of random variables Xn /Y has a limit with
probability 1. This limit is a random variable; it has the value 0 with probability F10 (1),
and has larger values with some given distribution. Intuitively, for large n, Xn is either 0
or very large. If it is very large, it tends to grow in an orderly way, increasing by a multiple
of Y in each subsequent generation. 210 5.3 CHAPTER 5. COUNTABLESTATE MARKOV CHAINS Birthdeath Markov chains A birthdeath Markov chain is a Markov chain in which the state space is the set of nonnegative integers; for all i ≥ 0, the transition probabilities satisfy Pi,i+1 > 0 and Pi+1,i > 0,
and for all i − j  > 1, Pij = 0 (see Figure 5.4). A transition from state i to i + 1 is regarded as
a birth and one from i + 1 to i as a death. Thus the restriction on the transition probabilities
means that only one birth or death can occur in one unit of time. Many applications of
birthdeath processes arise in queueing theory, where the state is the number of customers,
births are customer arrivals, and deaths are customer departures. The restriction to only
one arrival or departure at a time seems rather peculiar, but usually such a chain is a ﬁnely
sampled approximation to a continuous time process, and the time increments are then
small enough that multiple arrivals or departures in a time increment are unlikely and can
be ignored in the limit.
✿♥
✘0
②
❖ p0
q1 ③
♥
1
②
❖ p1
q2 ③
♥
2
②
❖ p2
q3 ③
♥
3
②
❖ p3
q4 ③
♥
4 ... ❖ 1 − p3 − q3 Figure 5.4: Birthdeath Markov chain. We denote Pi,i+1 by pi and Pi,i−1 by qi . Thus Pii = 1 − pi − qi . There is an easy way to
ﬁnd the steadystate probabilities of these birthdeath chains. In any sample function of
the process, note that the number of transitions from state i to i + 1 diﬀers by at most 1
from the number of transitions from i + 1 to i. If the process starts to the left of i and
ends to the right, then one more i → i + 1 transition occurs than i + 1 → i, etc. Thus if we
visualize a renewalreward process with renewals on occurrences of state i and unit reward
on transitions from state i to i + 1, the limiting timeaverage number of transitions per unit
time is πi pi . Similarly, the limiting timeaverage number of transitions per unit time from
i + 1 to i is πi+1 qi+1 . Since these two must be equal in the limit,
πi pi = πi+1 qi+1 for i ≥ 0. (5.27) The intuition in (5.27) is simply that the rate at which downward transitions occur from
i + 1 to i must equal the rate of upward transitions. Since this result is very important,
both here and in our later study of continuous time birthdeath processes, we show that
(5.27) also results from using the steadystate equations in (5.14):
πi = pi−1 πi−1 + (1 − pi − qi )πi + qi+1 πi+1 ; π0 = (1 − p0 )π0 + q1 π1 . i>0 (5.28)
(5.29) From (5.29), p0 π0 = q1 π1 . To see that (5.27) is satisﬁed for i > 0, we use induction on i,
with i = 0 as the base. Thus assume, for a given i, that pi−1 πi−1 = qi πi . Substituting this
in (5.28), we get pi πi = qi+1 πi+1 , thus completing the inductive proof. 5.4. REVERSIBLE MARKOV CHAINS 211 It is convenient to deﬁne ρi as pi /qi+1 . Then we have πi+1 = ρi πi , and iterating this,
πi = π0 i−1
Y ρj ; π0 = j =0 1+ P1 1 i=1 Qi−1 j =0 ρj . (5.30) PQ
If i≥1 0≤j <i ρj < 1, then π0 is positive and all the states are positiverecurrent. If this
sum of products is inﬁnite, then none of the states are positiverecurrent. If ρj is bounded
below 1, say ρj ≤ 1 − ≤ for some ﬁxed e > 0 and all suﬃciently l...
View Full
Document
 Spring '09
 R.Srikant

Click to edit the document details