{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Lecture 4_Presentation

# Lecture 4_Presentation - Observations thus far...

This preview shows pages 1–5. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Introductory Engineering Stochastic Processes, ORIE 3510 Instructor: Mark E. Lewis, Professor School of Operations Research and Information Engineering Cornell University Disclaimer : Notes are only meant as a lecture supplement not substitute! Instructor: Mark E. Lewis, Professor 1 Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Observations thus far We can use transition diagrams to read the transition matrix In studying long term behavior there are several considerations Can a state be visited over and over? In the limit ? If so, how often? Long-run frequency ? If not how many times will it be visited before never being visited again. Define transience and recurrence Visits to recurrent constitute “restarts”. Transition diagrams can help decide transience and recurrence Transience/recurrence can also be discerned via analysis (random walk example). Instructor: Mark E. Lewis, Professor 2 Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Definitions Communication as an Equivalence Relation Irreducibility (and Reducible Markov chains) Simple Examples Key Observation for Communication Accessibility and Communication In the previous (random walk) example we showed when the state zero is recurrent and when it is transient. What about the other states? Do we need to repeat the same analysis for each one? The main idea is that we divide the state space up into equivalence classes...and states within each class have the same transience/recurrence properties Instructor: Mark E. Lewis, Professor 3 Observations thus far Accessibility and Communication Decomposing the State Space (Reducible DTMCs) Main Points Definitions Communication as an Equivalence Relation Irreducibility (and Reducible Markov chains) Simple Examples Key Observation for Communication Accessibility and Communication Definition State j is said to be accessible from state i (written i...
View Full Document

{[ snackBarMessage ]}

### Page1 / 15

Lecture 4_Presentation - Observations thus far...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online