Markov_chainsIII-beamer

# Markov_chainsIII-beamer - Limiting Distributions Limiting...

This preview shows pages 1–4. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Limiting Distributions Limiting Probabilities for Transient States Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Stationary Distributions – Interpretations Computation of a Stationary Distribution Steady State Costs/Rewards Introductory Engineering Stochastic Processes, ORIE 3510 Instructor: Mark E. Lewis, Associate Professor School of Operations Research and Information Engineering Cornell University Disclaimer : Notes are only meant as a lecture supplement not substitute! 1/ 40 Limiting Distributions Limiting Probabilities for Transient States Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Stationary Distributions – Interpretations Computation of a Stationary Distribution Steady State Costs/Rewards Example – Outstanding Orders Definition Stationary Distribution Limiting Distribution as a Stationary Distribution Outstanding Orders The people at a large logistics company have been working for the same clients for almost a century Since the operations are somewhat static, it is safe to assume the system has been running smoothly for quite some time. You are asked to estimate the number of outstanding orders to be picked up on a typical day. Suppose we have a report with the first few years of historical data and can estimate p ij . How can we estimate the current number of outstanding orders? We could compute lim n →∞ p ( n ) ij ... if it exists (ahh, the plot thickens!) 2/ 40 Limiting Distributions Limiting Probabilities for Transient States Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Stationary Distributions – Interpretations Computation of a Stationary Distribution Steady State Costs/Rewards Example – Outstanding Orders Definition Stationary Distribution Limiting Distribution as a Stationary Distribution Limiting Distributions (intuition) Suppose I am interested in computing the probability we are in state j after 1 million and 1 periods. Starting in state i , what is the probability of being in state k after say, 1 million periods? p (1 million ) ik Two quick thoughts Although, there are no guarantees (it depends on the kind of state i is), one might expect that eventually, the initial state has less and less effect on this probability. Would you expect this probability to the much different if looked 1 million and 1 periods? So the probability of being in state j after a million and 1 periods is (call it π j (approximately)) π j = X k ∈ S π k p kj Purple says – “Condition on where you are at time 1 mil”, Green says – “take one more step”....
View Full Document

{[ snackBarMessage ]}

### Page1 / 40

Markov_chainsIII-beamer - Limiting Distributions Limiting...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online