{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Markov_chainsII_beamer

# Markov_chainsII_beamer - Introductory Engineering...

This preview shows pages 1–6. Sign up to view the full content.

Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor School of Operations Research and Information Engineering Cornell University Markov Chains – State Classifications 1/ 11

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Motivation In any system we model, we are interested in either the short-term or long-term behavior. There are two types of short-term behavior Behavior seen by states continuously revisited (recurrent), but before things “settle down” Behavior seen by states only finitely many times (transient) Long-term behavior is analyzed only for states continuously revisited...by looking at the relative frequency in which they are visited 2/ 11
Short-term (Transient) Behavior Suppose we have the following transition matrix P = 1 / 10 9 / 10 9 / 10 1 / 10 P 4 = 0 . 7048 0 . 2952 0 . 2952 0 . 7048 P 50 = 0 . 5000 0 . 5000 0 . 5000 0 . 5000 Both states are “recurrent”, but we might like to know how it behaves before settling down. 3/ 11

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Short-term (Transient) Behavior Consider a rat wandering in a maze. The maze has six rooms labeled F , 2 , 3 , 4 , 5 and S . If a room has k doors, the probability that the rat selects a particular door is 1 / k If the rat reaches room F (Food) or room S (shock), it stays there forever A transition diagram is in order. 4/ 11
Short-term (Transient) Behavior The transition matrix can be computed from the figure P = F 2 3 4 5 S 1 0 0 0 0 0 1 / 2 0 0 1 / 2 0 0 1 / 3 0 0 1 / 3 1 / 3 0 0 1 / 3 1 / 3 0 0 1 / 3 0 0 1 / 2 0 0 1 / 2 0 0 0 0 0 1 Two obvious questions 1 How much time is spent in each state before being “absorbed” into S or F?

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}