Markov_chainsIII_beamer

# Markov_chainsIII_beamer - Introductory Engineering...

This preview shows pages 1–11. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor School of Operations Research and Information Engineering Cornell University More on class Properties 1/ 24 Periodicity Now that we know that if the limiting distribution exists it is a stationary distribution, it remains to provide conditions under which the limit exists. Definition A state i in some Markov chain is said to have period d ( i ) if and only if d ( i ) is the largest integer such that p ( n ) ii = 0 whenever n is not divisible by d ( i ) . 2/ 24 Periodicity Now that we know that if the limiting distribution exists it is a stationary distribution, it remains to provide conditions under which the limit exists. Definition A state i in some Markov chain is said to have period d ( i ) if and only if d ( i ) is the largest integer such that p ( n ) ii = 0 whenever n is not divisible by d ( i ) . Definition An alternative definition is that the period of i is the greatest common divisor (gcd) of set of n for which p ( n ) ii > . 2/ 24 Examples Consider the following examples: P = 0 1 1 0 . In this case, since p ( n ) ii = ( 1 if n is even, if n is odd, lim n p ( n ) ii does not exist . The set of n for which p ( n ) ii > 0 is the set of even numbers; the gcd = 2. 3/ 24 Conclusion Conclusion: If the period is greater than 1 there is potential that the limiting distribution does not exist. 4/ 24 Aperiodicity Definition A state that has period 1 is said to be aperiodic. Note that if p ii > 0 then state i must be aperiodic. Why? 5/ 24 Another Example Consider now P = 0 1 / 2 1 / 2 0 1 1 1 Note that p (2) 00 > 0 and p (3) 00 > 0. Thus, the gcd is at most 1 (since only 1 divides 2 and 3 evenly). State zero is aperiodic. 6/ 24 Conclusion Conclusion: Since all states communicate, and zero is aperiodic the chain is aperiodic. If at least one state in an irreducible Markov chain is aperiodic then the whole chain is said to be aperiodic. 7/ 24 Another Example Consider now P = 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 There are two (positive) recurrent, aperiodic classes; { , 1 } and { 2 , 3 } . 8/ 24 Example Cont. Notice that P 2 = 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 1 / 2 = P ....
View Full Document

## This note was uploaded on 04/03/2008 for the course ORIE 361 taught by Professor Lewis,m. during the Spring '07 term at Cornell University (Engineering School).

### Page1 / 36

Markov_chainsIII_beamer - Introductory Engineering...

This preview shows document pages 1 - 11. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online