This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ORIE3510 Introduction to Engineering Stochastic Processes Spring 2010 Homework 3: Discrete Time Markov Chains Due 2:30pm, February 17, 2010 (drop box) Be sure to write your name and section number or day&time on your homework. In all questions, be sure to give the justification for your answers. There are 5 problems, each equally weighted for 100 total points. 1. In the following two cases give the classes, and determine which states are transient and which are recurrent. (a) S = { , 1 , 2 , 3 , 4 , 5 } and the transition probability matrix is P = 1 3 2 3 1 4 3 4 2 3 1 3 1 5 4 5 1 4 1 4 1 4 1 4 1 6 1 6 1 6 1 6 1 6 1 6 (b) S = { 1 ,..., 9 } and the transition probability matrix is ( x signifies a positive entry) P = 0 0 0 x 0 0 0 0 x x x x 0 0 0 x 0 0 0 0 0 0 0 x x 0 0 0 0 0 0 0 0 0 0 0 0 x 0 0 0 0 x 0 0 0 0 0 0 0 x 0 0 0 x x 0 0 0 0 x 0 0 0 0 0 0 0 0 0...
View
Full Document
 Spring '10
 LEWIS
 Stochastic process, Markov chain, transition probability matrix

Click to edit the document details