This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: IEOR 4701: HMWK 2 1. Consider a Markov chain with state space S = { , 1 , 2 , 3 } with transition matrix P = 1 / 2 1 / 8 1 / 8 1 / 4 1 / 2 1 / 2 1 / 6 1 / 3 1 / 3 1 / 6 1 / 2 1 / 2 . (a) Find P (2) 3 , 1 = P ( X 2 = 1  X = 3) in two ways: (1) By direct calculation via considering the paths which would take the chain 3 → 1 in two steps (such as 3 → 1 → 1 which ocurrs with probability P 3 , 1 P 1 , 1 = (1 / 2)(1 / 2) = 1 / 4). (2) by using the fact that P (2) = P 2 . (b) Show that all states communicate, hence the chain has only one communication class, C = S = { , 1 , 2 , 3 } hence is irreducible . (c) Solve π = πP for the limiting distribution π = ( π ,π 1 ,π 2 ,π 3 ), where π j > , j ∈ S , and ∑ j ∈S π j = 1. (d) If X n denotes the amount of bonus money you earn during the n th year of your job (in units of 10,000) then, on average (over all time) what is your average bonus?...
View
Full
Document
This note was uploaded on 10/16/2010 for the course IEOR 4701 taught by Professor Karlsigma during the Summer '10 term at Columbia.
 Summer '10
 KarlSigma

Click to edit the document details