{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

hw2 - HW 2 Practice with DTMCs CS/EE 147 Assigned Guru Lina...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
CS/EE 147 HW 2: Practice with DTMCs Guru: Lina Assigned: 04/06/10 Due: 04/16/10, Raga’s mailbox, 1pm We encourage you to discuss these problems with others, but you need to write up the actual solutions alone. At the top of your homework sheet, list all the people with whom you discussed. Crediting help from other classmates will not take away any credit from you. Start early and come to office hours with your questions! Note before you begin This is a long and probably hard problem set. Start early – do not put it off until the last minute! 1 DTMC Warmup [20 points] (a) For each of the following state whether the chain is irreducible, aperiodic, and/or positive recurrent. (i) 1 4 1 4 1 2 0 0 1 1 0 0 (ii) 0 1 0 0 1 0 1 0 0 (iii) 1 3 0 2 3 1 4 3 4 0 0 0 1 (b) Consider the following probability transition matrices P (1) = 0 2 / 3 0 1 / 3 1 / 3 0 2 / 3 0 0 1 / 3 0 2 / 3 2 / 3 0 1 / 3 0 P (2) = 1 / 3 2 / 3 0 0 1 / 3 0 2 / 3 0 0 1 / 3 0 2 / 3 0 0 1 / 3 2 / 3 For each of these, (i) Draw the corresponding Markov chains. (ii) Solve for the stationary probabilities. First, try to use the local balance equations, and if they don’t work use the balance equations. (iii) For those chain(s) where the local balance equations worked, explain why it makes sense that for all states i , j in the chain, the rate of transitions from i to j should equal the rate of transitions from j to i . 2 Finite state DTMCs [15 points] (a) Prove the following two class property theorems. (i) Null-recurrence is a class property, i.e., if i is null-recurrent and i communicates with j then j is null-recurrent.
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}