This preview shows pages 1–6. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Introductory Engineering Stochastic Processes, ORIE 361 Instructor: Mark E. Lewis, Associate Professor School of Operations Research and Information Engineering Cornell University Markov Chains State Classifications 1/ 11 Motivation In any system we model, we are interested in either the shortterm or longterm behavior. There are two types of shortterm behavior Behavior seen by states continuously revisited (recurrent), but before things settle down Behavior seen by states only finitely many times (transient) Longterm behavior is analyzed only for states continuously revisited...by looking at the relative frequency in which they are visited 2/ 11 Shortterm (Transient) Behavior Suppose we have the following transition matrix P = 1 / 10 9 / 10 9 / 10 1 / 10 P 4 = . 7048 0 . 2952 . 2952 0 . 7048 P 50 = . 5000 0 . 5000 . 5000 0 . 5000 Both states are recurrent, but we might like to know how it behaves before settling down. 3/ 11 Shortterm (Transient) Behavior Consider a rat wandering in a maze. The maze has six rooms labeled F , 2 , 3 , 4 , 5 and S . If a room has k doors, the probability that the rat selects a particular door is 1 / k If the rat reaches room F (Food) or room S (shock), it stays there forever A transition diagram is in order. 4/ 11 Shortterm (Transient) Behavior The transition matrix can be computed from the figure P = F 2 3 4 5 S 1 1 / 2 1 / 2 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 3 1 / 2 1 / 2 1 Two obvious questions 1 How much time is spent in each state before being absorbed into S or F?into S or F?...
View Full
Document
 Spring '07
 LEWIS,M.

Click to edit the document details