Markov

# Now suppose that p n j j 0 then p n1 nn2 i i 0 so

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: at both states 1 and 2 in the example P = 0 1 have period 2. In fact, the 10 next result shows that if two states i and j communicate, then they must have the same period. 1.30 Theorem. If the states i and j communicate, then di = dj . Proof: Since j is accessible from i, by 1.27 there exists an n1 such that P n1 i; j  P n2 j; i 0. 0. Noting that Similarly, since i is accessible from j , there is an n2 such that Pn1 +n2 i; i 0, it follows that di j n1 + n2; that is, di divides n1 + n2 , which means that n1 + n2 is an integer multiple of di . Now suppose that P n j; j  0. Then P n1 +n+n2 i; i 0, so that di j n1 + n + n2: Stochastic Processes J. Chang, March 30, 1999 1.6. IRREDUCIBILITY, PERIODICITY, AND RECURRENCE Page 1-15 Subtracting the last two displays gives di j n. Since n was an arbitrary integer satisfying P n j; j  0, we have found that di is a common divisor of the set fn : P nj; j  0g. Since dj is de ned to be the greatest common divisor of this set, we have shown that dj  di . Interchanging the roles of i and j in the previous argument gives the opposite inequality di  dj . This completes the proof. It follows from Theorem 1.30 that all states in a communicating class have the same period. We say that the period of a state is a class property." In particular, all states in an irreducible Markov chain have the same period. Thus, we can speak of the period of a Markov chain if that Markov chain is irreducible: the period of an irreducible Markov chain is the period of any of its states. 1.31 Definition. An irreducible Markov chain is said to be aperiodic if its period is 1, and periodic otherwise. We have now discussed all of the words we need in order to understand the statement of the Basic Limit Theorem 1.17. We will need another concept or two before we can get to the proof, and the proof will then take some time beyond that. So I propose that we pause to discuss an interesting example of an application of the Basic Limit Theorem; this will help us build up some motivation to help carry us through the proof, and will also give some practice that should help be helpful in assimilating the concepts of irreducibility and aperiodicity. 1.32 Example Generating a random table with fixed row and column sums . Consider the 4  4 table of numbers that is enclosed within the rectangle below. The four numbers along the bottom of the table are the column sums, and those along the right edge of the table are the row sums. 68 119 26 7 220 20 84 17 94 215 15 54 14 10 93 5 29 14 16 64 108 286 71 127 Stochastic Processes J. Chang, March 30, 1999 Page 1-16 1. MARKOV CHAINS Suppose we want to generate a random 4  4 table that has the same row and column sums as the table above. That is, suppose that we want to generate a random table of nonnegative integers whose probability distribution is uniform on the set S of all such 4  4 tables that have the given row and column sums. Here is a proposed algorithm. Start with any table having the correct row and column sums; so of course the 4  4 table given above will do. Denote the entries in , ...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online