Markov_chainsIV-beamer

# Markov_chainsIV-beamer - Long Term Behavior for Transient...

This preview shows pages 1–5. Sign up to view the full content.

Long Term Behavior for Transient States Example – Nice City The Expected Number of Visits to Each Transient State Probability of Ever Reaching a State Absorption Probabilities The Nice City Example with Numbers Introductory Engineering Stochastic Processes, ORIE 3510 Instructor: Professor Mark E. Lewis School of Operations Research and Information Engineering Cornell University Disclaimer : Notes are only meant as a lecture supplement not substitute! 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Long Term Behavior for Transient States Example – Nice City The Expected Number of Visits to Each Transient State Probability of Ever Reaching a State Absorption Probabilities The Nice City Example with Numbers Long Term Behavior for Transient States Recall that the long-term behavior, starting in a particular state i is captured by lim n →∞ p ( n ) ij . When j is transient, this limit is zero. When j is recurrent, we need to know the probability that when the DTMC enters a recurrent class (starting in i ), that said class is the one that contains j . What about what happens. ..along the way? For example, how much time is spent in each transient state before entering ( any ) recurrent class? Or, if there is another particular (transient) state of interest, what is the probability of ever entering that state? 2
Long Term Behavior for Transient States Example – Nice City The Expected Number of Visits to Each Transient State Probability of Ever Reaching a State Absorption Probabilities The Nice City Example with Numbers Questions of Interest The Transition Matrix Rewriting the matrix Grand Theft Auto NICE City To get a property you must commit some acts of kindness at that property Your strategy is to work on two properties You have chosen to randomize slightly: you will work on one property and then move to working on the other property with some ﬁxed probabilities While working on one property, there is the possibility that you will ﬁgure out how to complete the task for the other property, complete it and be done Once either task is completed (corresponding to a property), you own the property. ..and it generates revenue You then sit back. ..and collect your money 3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Long Term Behavior for Transient States Example – Nice City The Expected Number of Visits to Each Transient State Probability of Ever Reaching a State Absorption Probabilities The Nice City Example with Numbers Questions of Interest
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 02/12/2012 for the course ORIE 3510 taught by Professor Resnik during the Spring '09 term at Cornell.

### Page1 / 16

Markov_chainsIV-beamer - Long Term Behavior for Transient...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online