STA3007_0506_t06_ssol

STA3007_0506_t06_ssol - 3 ) 2 + ( 1 3 ) 3 + ... = 1 2...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
STA 3007 Applied Probability 2005 Tutorial 6 Suggested Solution 1. The Long Run Behavior of Markov Chains i. Since 0 1 and 1 2, so 0 2. All states can communicate with each other, the Markov chain is irreducible. ii. 4 5 and 5 2, so 4 2 2 3 and 3 4, so 2 4 Therefore, 2 4 4 5 and 5 3, so 4 3 3 4, therefore, 3 4 3 4 and 4 5, so 3 5 5 3, therefore, 3 5 Therefore, states 2, 3, 4 and 5 can communicate. The classes are: { 0 } , { 1 } , { 2 , 3 , 4 , 5 } d (0) = GCD { 1 , 2 , 3 ,... } = 1 d (1) = 0 d (5) = d (2) = d (3) = d (4) = 1 iii. The classes are: { 0 } , { 1 } , { 2 , 4 } , { 3 } , { 5 } Method 1: f 00 = n =0 f ( n ) 00 = 1 3 < 1 Therefore, State 0 is a transient state. f 11 = n =0 f ( n ) 11 = 1 4 < 1 Therefore, State 1 is a transient state. f 22 = n =0 f ( n ) 22 = 0 + 1(1) = 1 Therefore, State 2 and 4 are recurrent states since they are in the same class. f 33 = n =0 f ( n ) 33 = 0 < 1 Therefore, State 3 is a transient state. f 55 = n =0 f ( n ) 55 = 1 Therefore, State 5 is a recurrent state. Method 2: n =0 P ( n ) 00 = 1 3 + ( 1
Background image of page 1
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 3 ) 2 + ( 1 3 ) 3 + ... = 1 2 &lt; ∞ Therefore, State 0 is a transient state. ∑ ∞ n =0 P ( n ) 11 = 1 3 &lt; ∞ Therefore, State 1 is a transient state. ∑ ∞ n =0 P ( n ) 22 = 0 + 1 + 0 + 1 + ... = ∞ Therefore, State 2 and 4 are recurrent states since they are in the same class. ∑ ∞ n =0 P ( n ) 33 = 0 &lt; ∞ Therefore, State 3 is a transient state. ∑ ∞ n =0 P ( n ) 55 = 1 + 1 + 1 + 1 + ... = ∞ Therefore, State 5 is a recurrent state. iv. Given P (0) 00 = 1 ,f 00 = 0 Finding out P (2) 00 = 0 . 25 ,P (3) 00 = 0 . 125 ,P (4) 00 = 0 . 375 ,P (5) 00 = 0 . 2188 By solving P ( n ) ii = ∑ n k =0 f ( k ) ii P ( n-k ) ii , n = 1 ,..., 5 f (1) 00 = 0 ,f (2) 00 = 0 . 25 ,f 3) 00 = 0 . 125 ,f (4) 00 = 0 . 3125 ,f (5) 00 = 0 . 1563 1...
View Full Document

Ask a homework question - tutors are online