{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Markov_chains_together_beamer

# Markov_chains_together_beamer - Introductory Engineering...

This preview shows pages 1–10. Sign up to view the full content.

Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor School of Operations Research and Information Engineering Cornell University A Complete Finite State Example 1/ 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor DTMC Example Consider a DTMC with the following one-step transition matrix. P = 1 2 3 4 5 6 7 8 9 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 / 3 1 / 3 0 0 0 0 0 0 0 1 / 2 1 / 2 0 0 0 0 0 0 1 / 2 0 0 0 1 / 2 0 0 0 2 / 3 0 0 0 1 / 3 0 0 0 0 0 1 / 2 1 / 2 0 0 0 0 0 0 0 0 0 0 0 0 1 / 4 3 / 4 2/ 1
Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Transition Diagram 5 2 3 6 7 4 8 9 1 1 1 1 1/2 1/2 2/3 1/3 1/4 3/4 1/3 1/2 1/2 2/3 1/2 1/2 3/ 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Transition Diagram 5 2 3 6 7 4 8 9 1 1 1 1 1/2 1/2 2/3 1/3 1/4 3/4 1/3 1/2 1/2 2/3 1/2 1/2 3/ 1
Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Classes I Recurrent: { 1 , 2 , 3 } , { 4 , 5 } I Transient: { 6 , 7 } , { 8 } , { 9 } I Suppose f ( i ) = i for i = 1 , 2 , . . . , 9 4/ 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Classes I Recurrent: { 1 , 2 , 3 } , { 4 , 5 } I Transient: { 6 , 7 } , { 8 } , { 9 } I Suppose f ( i ) = i for i = 1 , 2 , . . . , 9 I Hopefully by now we know that we know how to compute E ( f ( X 3 ) | X 0 = i ) 4/ 1
Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Classes I Recurrent: { 1 , 2 , 3 } , { 4 , 5 } I Transient: { 6 , 7 } , { 8 } , { 9 } I Suppose f ( i ) = i for i = 1 , 2 , . . . , 9 I Hopefully by now we know that we know how to compute E ( f ( X 3 ) | X 0 = i ) I Compute P 3 by matrix multiplication 4/ 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Classes I Recurrent: { 1 , 2 , 3 } , { 4 , 5 } I Transient: { 6 , 7 } , { 8 } , { 9 } I Suppose f ( i ) = i for i = 1 , 2 , . . . , 9 I Hopefully by now we know that we know how to compute E ( f ( X 3 ) | X 0 = i ) I Compute P 3 by matrix multiplication I This yields p (3) ij for all i , j S . 4/ 1
Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor Classes I Recurrent: { 1 , 2 , 3 } , { 4 , 5 } I Transient: { 6 , 7 } , { 8 } , { 9 } I Suppose f ( i ) = i for i = 1 , 2 , . . . , 9 I Hopefully by now we know that we know how to compute E ( f ( X 3 ) | X 0 = i ) I Compute P 3 by matrix multiplication I

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 28

Markov_chains_together_beamer - Introductory Engineering...

This preview shows document pages 1 - 10. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online