This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: game without slipping back to square 1? (Hint: Model the game as a Markov Chain, and calculate μ 19 .) (Superhint: Do you really need 9 states in your model?) Problem 9 is an optional question, those who solve it will get extra credits. 9. Let P n × n be the transition matrix of a markov chain with a Fnite state space S = { 1 , 2 , . . . n } . a) Show that Π 1 × n is the stationary distribution of the Markov chain if and only if Π ( IP + A ) = a , where A = ( a ij : i, j ∈ S ) with a ij = 1 for all i and j . a 1 × n = ( a i : i ∈ S ) with a i = 1 for all i . and I n × n is the identity matrix. b) Show that if the Markov chain is irreducible, then ( I + PA ) is invertible. Note that now we can compute Π by simply inverting a matrix, that is Π = a ( IP + A )1 Note : 4.8 (4.7) implies problem 4.8 from the ninth edition which is the same as 4.7 in the eighth edition. 2...
View
Full Document
 Fall '04
 Chow
 Markov chain, three days, finite state space

Click to edit the document details