This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: ASOC ACTL2003 Support Notes: Week 2 written by Tim Yip, Andy Wong and Andrew Teh 0.1 Markov Chains 0.1.1 Periodicity Definition: State i has period d , where d is the greatest common divisor for all n for which P n ii > . For example, if P n ii > for n = 2 , 4 , 5 , then d = gcd (2 , 4 , 5) = 1 . A state with period 1 is aperiodic Two states that communicate have the same period A recurrent state that has expected time of return to itself that is finite is positive recurrent If the Markov Chain has a finite number of states (this is generally the case in this course), all recurrent states are positive recurrent A state that is positive recurrent and aperiodic is ergodic 0.1.2 Limiting Probabilities The limiting probability is defined as j = lim n P n ij As the notation suggests, it is independent of i . This limiting probability exists if the Markov Chain is irreducible and ergodic....
View Full Document
This note was uploaded on 06/12/2011 for the course ASB 2003 taught by Professor Kim during the Three '11 term at University of New South Wales.
- Three '11