hw2 - HW 2: Practice with DTMCs CS/EE 147 Assigned:...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
HW 2: Practice with DTMCs Guru: Lina Assigned: 04/06/10 Due: 04/16/10, Raga’s mailbox, 1pm We encourage you to discuss these problems with others, but you need to write up the actual solutions alone. At the top of your homework sheet, list all the people with whom you discussed. Crediting help from other classmates will not take away any credit from you. Start early and come to office hours with your questions! Note before you begin This is a long and probably hard problem set. Start early – do not put it off until the last minute! 1 DTMC Warmup [20 points] (a) For each of the following state whether the chain is irreducible, aperiodic, and/or positive recurrent. (i) 1 4 1 4 1 2 0 0 1 1 0 0 (ii) 0 1 0 0 1 0 1 0 0 (iii) 1 3 0 2 3 1 4 3 4 0 0 0 1 (b) Consider the following probability transition matrices P (1) = 0 2 / 3 0 1 / 3 1 / 3 0 2 / 3 0 0 1 / 3 0 2 / 3 2 / 3 0 1 / 3 0 P (2) = 1 / 3 2 / 3 0 0 1 / 3 0 2 / 3 0 0 1 / 3 0 2 / 3 0 0 1 / 3 2 / 3 For each of these, (i) Draw the corresponding Markov chains. (ii) Solve for the stationary probabilities. First, try to use the local balance equations, and if they don’t work use the balance equations. (iii) For those chain(s) where the local balance equations worked, explain why it makes sense that for all states i , j in the chain, the rate of transitions from i to j should equal the rate of transitions from j to i . 2 Finite state DTMCs [15 points] (a) Prove the following two class property theorems. (i) Null-recurrence is a class property, i.e., if
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 01/05/2012.

Page1 / 3

hw2 - HW 2: Practice with DTMCs CS/EE 147 Assigned:...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online