{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Markov_chainsII_beamer

Markov_chainsII_beamer - Introductory Engineering...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor School of Operations Research and Information Engineering Cornell University Markov Chains – State Classifications 1/ 11
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Motivation In any system we model, we are interested in either the short-term or long-term behavior. There are two types of short-term behavior Behavior seen by states continuously revisited (recurrent), but before things “settle down” Behavior seen by states only finitely many times (transient) Long-term behavior is analyzed only for states continuously revisited...by looking at the relative frequency in which they are visited 2/ 11
Background image of page 2
Short-term (Transient) Behavior Suppose we have the following transition matrix P = 1 / 10 9 / 10 9 / 10 1 / 10 P 4 = 0 . 7048 0 . 2952 0 . 2952 0 . 7048 P 50 = 0 . 5000 0 . 5000 0 . 5000 0 . 5000 Both states are “recurrent”, but we might like to know how it behaves before settling down. 3/ 11
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Short-term (Transient) Behavior Consider a rat wandering in a maze. The maze has six rooms labeled F , 2 , 3 , 4 , 5 and S . If a room has k doors, the probability that the rat selects a particular door is 1 / k If the rat reaches room F (Food) or room S (shock), it stays there forever A transition diagram is in order. 4/ 11
Background image of page 4
Short-term (Transient) Behavior The transition matrix can be computed from the figure P = F 2 3 4 5 S 1 0 0 0 0 0 1 / 2 0 0 1 / 2 0 0 1 / 3 0 0 1 / 3 1 / 3 0 0 1 / 3 1 / 3 0 0 1 / 3 0 0 1 / 2 0 0 1 / 2 0 0 0 0 0 1 Two obvious questions 1 How much time is spent in each state before being “absorbed” into S or F?
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}