This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Limiting Distributions Limiting Probabilities for Transient States Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Stationary Distributions – Interpretations Computation of a Stationary Distribution Steady State Costs/Rewards Introductory Engineering Stochastic Processes, ORIE 3510 Instructor: Mark E. Lewis, Associate Professor School of Operations Research and Information Engineering Cornell University Disclaimer : Notes are only meant as a lecture supplement not substitute! 1/ 40 Limiting Distributions Limiting Probabilities for Transient States Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Stationary Distributions – Interpretations Computation of a Stationary Distribution Steady State Costs/Rewards Example – Outstanding Orders Definition Stationary Distribution Limiting Distribution as a Stationary Distribution Outstanding Orders The people at a large logistics company have been working for the same clients for almost a century Since the operations are somewhat static, it is safe to assume the system has been running smoothly for quite some time. You are asked to estimate the number of outstanding orders to be picked up on a typical day. Suppose we have a report with the first few years of historical data and can estimate p ij . How can we estimate the current number of outstanding orders? We could compute lim n →∞ p ( n ) ij ... if it exists (ahh, the plot thickens!) 2/ 40 Limiting Distributions Limiting Probabilities for Transient States Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Stationary Distributions – Interpretations Computation of a Stationary Distribution Steady State Costs/Rewards Example – Outstanding Orders Definition Stationary Distribution Limiting Distribution as a Stationary Distribution Limiting Distributions (intuition) Suppose I am interested in computing the probability we are in state j after 1 million and 1 periods. Starting in state i , what is the probability of being in state k after say, 1 million periods? p (1 million ) ik Two quick thoughts Although, there are no guarantees (it depends on the kind of state i is), one might expect that eventually, the initial state has less and less effect on this probability. Would you expect this probability to the much different if looked 1 million and 1 periods? So the probability of being in state j after a million and 1 periods is (call it π j (approximately)) π j = X k ∈ S π k p kj Purple says – “Condition on where you are at time 1 mil”, Green says – “take one more step”....
View
Full Document
 Spring '09
 RESNIK
 Thermodynamics, Steady State, Markov chain, stationary distribution, infinite state spaces, steady state equations

Click to edit the document details