This preview shows pages 1–6. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Transition Matrix Transition Diagram Classes and Short Term Calculations LongTerm Behavior Longterm Behavior Recurrent States Average Costs Introductory Engineering Stochastic Processes, ORIE 3510 Instructor: Professor Mark E. Lewis School of Operations Research and Information Engineering Cornell University Disclaimer : Notes are only meant as a lecture supplement not substitute! 1 Transition Matrix Transition Diagram Classes and Short Term Calculations LongTerm Behavior Longterm Behavior Recurrent States Average Costs DTMC Example Consider a DTMC with the following onestep transition matrix. P = 1 2 3 4 5 6 7 8 9 1 1 1 2 / 3 1 / 3 1 / 2 1 / 2 1 / 2 1 / 2 0 2 / 3 1 / 3 1 / 2 1 / 2 1 / 4 3 / 4 2 Transition Matrix Transition Diagram Classes and Short Term Calculations LongTerm Behavior Longterm Behavior Recurrent States Average Costs Transition Diagram 5 2 3 6 7 4 8 9 1 1 1 1 1/2 1/2 2/3 1/3 1/4 3/4 1/3 1/2 1/2 2/3 1/2 1/2 5 2 3 6 7 4 8 9 1 1 1 1 1/2 1/2 2/3 1/3 1/2 1/4 3/4 1/3 1/2 1/2 2/3 1/2 3 Transition Matrix Transition Diagram Classes and Short Term Calculations LongTerm Behavior Longterm Behavior Recurrent States Average Costs Classes and Short Term Calculations Recurrent: { 1 , 2 , 3 } , { 4 , 5 } Transient: { 6 , 7 } , { 8 } , { 9 } Suppose f ( i ) = i for i = 1 , 2 ,..., 9 Hopefully by now we know that we know how to compute E ( f ( X 3 )  X = i ) Compute P 3 by matrix multiplication This yields p (3) ij for all i , j S . Compute E ( f ( X 3 )  X = i ) = j S f ( j ) p (3) ij (this is 1.3891 for i = 7) 4 Transition Matrix Transition Diagram Classes and Short Term Calculations LongTerm Behavior Longterm Behavior Recurrent States Average Costs Rearrange the state space Cost Until Absorption Probability of Ever Reaching j Probability of Being Absorbed Into...
View Full
Document
 Spring '09
 RESNIK

Click to edit the document details