This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Introductory Engineering Stochastic Processes, ORIE 3510 Instructor: Mark E. Lewis, Professor School of Operations Research and Information Engineering Cornell University Disclaimer : Notes are only meant as a lecture supplement not substitute! Instructor: Mark E. Lewis, Professor 1 Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Observations thus far We can use transition diagrams to read the transition matrix In studying long term behavior there are several considerations Can a state be visited over and over? In the limit ? If so, how often? Longrun frequency ? If not how many times will it be visited before never being visited again. Define transience and recurrence Visits to recurrent constitute “restarts”. Transition diagrams can help decide transience and recurrence Transience/recurrence can also be discerned via analysis (random walk example). Instructor: Mark E. Lewis, Professor 2 Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Definitions Communication as an Equivalence Relation Irreducibility (and Reducible Markov chains) Simple Examples Key Observation for Communication Accessibility and Communication In the previous (random walk) example we showed when the state zero is recurrent and when it is transient. What about the other states? Do we need to repeat the same analysis for each one? The main idea is that we divide the state space up into equivalence classes...and states within each class have the same transience/recurrence properties Instructor: Mark E. Lewis, Professor 3 Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Definitions Communication as an Equivalence Relation Irreducibility (and Reducible Markov chains) Simple Examples Key Observation for Communication Accessibility and Communication Definition State j is said to be accessible from state i (written i...
View
Full Document
 Spring '09
 RESNIK
 Markov chain, Equivalence relation, Mark E. Lewis

Click to edit the document details