Markov_chainsII_beamer - Introductory Engineering...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor School of Operations Research and Information Engineering Cornell University Markov Chains State Classifications 1/ 11 Motivation In any system we model, we are interested in either the short-term or long-term behavior. There are two types of short-term behavior Behavior seen by states continuously revisited (recurrent), but before things settle down Behavior seen by states only finitely many times (transient) Long-term behavior is analyzed only for states continuously revisited...by looking at the relative frequency in which they are visited 2/ 11 Short-term (Transient) Behavior Suppose we have the following transition matrix P = 1 / 10 9 / 10 9 / 10 1 / 10 P 4 = . 7048 0 . 2952 . 2952 0 . 7048 P 50 = . 5000 0 . 5000 . 5000 0 . 5000 Both states are recurrent, but we might like to know how it behaves before settling down. 3/ 11 Short-term (Transient) Behavior Consider a rat wandering in a maze. The maze has six rooms labeled F , 2 , 3 , 4 , 5 and S . If a room has k doors, the probability that the rat selects a particular door is 1 / k If the rat reaches room F (Food) or room S (shock), it stays there forever A transition diagram is in order. 4/ 11 Short-term (Transient) Behavior The transition matrix can be computed from the figure P = F 2 3 4 5 S 1 1 / 2 1 / 2 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 2 1 / 2 1 Two obvious questions 1 How much time is spent in each state before being absorbed into S or F?into S or F?...
View Full Document

Page1 / 17

Markov_chainsII_beamer - Introductory Engineering...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online