This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Math331, Fall 2008 Instructor: David Anderson 2nd Markov Chain Homework 1. Consider a Markov chain transition matrix P = 1 / 2 1 / 3 1 / 6 3 / 4 1 / 4 1 . (a) Show the this is a regular Markov chain. (b) If the process is started in state 1, find the probability that it is in state 3 after two steps. (c) Find the limiting probability vector w . (d) Find lim n →∞ p n ( x, y ) for all x, y ∈ S . Why do you know these limits exist? Solution: (a) We have that P 3 = 1 / 2 1 / 3 1 / 6 9 16 1 / 4 3 / 16 3 / 8 1 / 2 1 / 8 and so the Markov chain is regular. (b) We have P 2 = 1 / 2 1 / 3 1 / 6 3 / 8 1 / 2 1 / 8 3 / 4 1 / 4 . Thus P ( X 2 = 3  X = 1) = P 2 (1 , 3) = 1 / 6. (c) We have to solve the system of equations xP = x = ⇒ [ x 1 , x 2 , x 3 ] 1 / 2 1 / 3 1 / 6 3 / 4 1 / 4 1 = [ x 1 , x 2 , x 3 ] ....
View
Full
Document
This note was uploaded on 01/05/2010 for the course ECE 01 taught by Professor All during the Spring '09 term at Aarhus Universitet.
 Spring '09
 all

Click to edit the document details