Markov_chains_together_beamer - Introductory Engineering...

Info iconThis preview shows pages 1–9. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor School of Operations Research and Information Engineering Cornell University A Complete Finite State Example 1/ 1 Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor DTMC Example Consider a DTMC with the following one-step transition matrix. P = 1 2 3 4 5 6 7 8 9 1 1 1 2 / 3 1 / 3 1 / 2 1 / 2 1 / 2 1 / 2 0 2 / 3 1 / 3 1 / 2 1 / 2 1 / 4 3 / 4 2/ 1 Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Transition Diagram 5 2 3 6 7 4 8 9 1 1 1 1 1/2 1/2 2/3 1/3 1/4 3/4 1/3 1/2 1/2 2/3 1/2 1/2 3/ 1 Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Transition Diagram 5 2 3 6 7 4 8 9 1 1 1 1 1/2 1/2 2/3 1/3 1/4 3/4 1/3 1/2 1/2 2/3 1/2 1/2 5 2 3 6 7 4 8 9 1 1 1 1 1/2 1/2 2/3 1/3 1/2 1/4 3/4 1/3 1/2 1/2 2/3 1/2 3/ 1 Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Classes I Recurrent: { 1 , 2 , 3 } , { 4 , 5 } I Transient: { 6 , 7 } , { 8 } , { 9 } I Suppose f ( i ) = i for i = 1 , 2 ,..., 9 4/ 1 Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Classes I Recurrent: { 1 , 2 , 3 } , { 4 , 5 } I Transient: { 6 , 7 } , { 8 } , { 9 } I Suppose f ( i ) = i for i = 1 , 2 ,..., 9 I Hopefully by now we know that we know how to compute E ( f ( X 3 ) | X = i ) 4/ 1 Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Classes I Recurrent: { 1 , 2 , 3 } , { 4 , 5 } I Transient: { 6 , 7 } , { 8 } , { 9 } I Suppose f ( i ) = i for i = 1 , 2 ,..., 9 I Hopefully by now we know that we know how to compute E ( f ( X 3 ) | X = i ) I Compute P 3 by matrix multiplication 4/ 1 Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Classes I Recurrent: { 1 , 2 , 3 } , { 4 , 5 } I Transient: { 6 , 7 } , { 8 } , { 9 } I Suppose f ( i ) = i for i = 1 , 2 ,..., 9 I Hopefully by now we know that we know how to compute E ( f ( X...
View Full Document

This note was uploaded on 04/03/2008 for the course ORIE 361 taught by Professor Lewis,m. during the Spring '07 term at Cornell University (Engineering School).

Page1 / 28

Markov_chains_together_beamer - Introductory Engineering...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online