Homework 2 - IEOR 4701 HMWK 2 1 Consider a Markov chain...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: IEOR 4701: HMWK 2 1. Consider a Markov chain with state space S = { , 1 , 2 , 3 } with transition matrix P = 1 / 2 1 / 8 1 / 8 1 / 4 1 / 2 1 / 2 1 / 6 1 / 3 1 / 3 1 / 6 1 / 2 1 / 2 . (a) Find P (2) 3 , 1 = P ( X 2 = 1 | X = 3) in two ways: (1) By direct calculation via considering the paths which would take the chain 3 → 1 in two steps (such as 3 → 1 → 1 which ocurrs with probability P 3 , 1 P 1 , 1 = (1 / 2)(1 / 2) = 1 / 4). (2) by using the fact that P (2) = P 2 . (b) Show that all states communicate, hence the chain has only one communication class, C = S = { , 1 , 2 , 3 } hence is irreducible . (c) Solve π = πP for the limiting distribution π = ( π ,π 1 ,π 2 ,π 3 ), where π j > , j ∈ S , and ∑ j ∈S π j = 1. (d) If X n denotes the amount of bonus money you earn during the n th year of your job (in units of 10,000) then, on average (over all time) what is your average bonus?...
View Full Document

This note was uploaded on 10/16/2010 for the course IEOR 4701 taught by Professor Karlsigma during the Summer '10 term at Columbia.

Page1 / 2

Homework 2 - IEOR 4701 HMWK 2 1 Consider a Markov chain...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online