Lecture 4_Presentation - Observations thus far...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Introductory Engineering Stochastic Processes, ORIE 3510 Instructor: Mark E. Lewis, Professor School of Operations Research and Information Engineering Cornell University Disclaimer : Notes are only meant as a lecture supplement not substitute! Instructor: Mark E. Lewis, Professor 1 Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Observations thus far We can use transition diagrams to read the transition matrix In studying long term behavior there are several considerations Can a state be visited over and over? In the limit ? If so, how often? Long-run frequency ? If not how many times will it be visited before never being visited again. Define transience and recurrence Visits to recurrent constitute restarts. Transition diagrams can help decide transience and recurrence Transience/recurrence can also be discerned via analysis (random walk example). Instructor: Mark E. Lewis, Professor 2 Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Definitions Communication as an Equivalence Relation Irreducibility (and Reducible Markov chains) Simple Examples Key Observation for Communication Accessibility and Communication In the previous (random walk) example we showed when the state zero is recurrent and when it is transient. What about the other states? Do we need to repeat the same analysis for each one? The main idea is that we divide the state space up into equivalence classes...and states within each class have the same transience/recurrence properties Instructor: Mark E. Lewis, Professor 3 Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Definitions Communication as an Equivalence Relation Irreducibility (and Reducible Markov chains) Simple Examples Key Observation for Communication Accessibility and Communication Definition State j is said to be accessible from state i (written i...
View Full Document

This note was uploaded on 02/12/2012 for the course ORIE 3510 taught by Professor Resnik during the Spring '09 term at Cornell University (Engineering School).

Page1 / 15

Lecture 4_Presentation - Observations thus far...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online