This preview shows pages 1–6. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Where are we so far? Limiting Distributions Limiting Probabilities Dependent on the Starting State Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Introductory Engineering Stochastic Processes, ORIE 3510 Instructor: Professor Mark E. Lewis School of Operations Research and Information Engineering Cornell University Disclaimer : Notes are only meant as a lecture supplement not substitute! 1 Where are we so far? Limiting Distributions Limiting Probabilities Dependent on the Starting State Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Where are we so far? States that we visit infinitely many times (given that we start in said state) are called recurrent . Otherwise they are called transient . Accessibility leads to communication and the creation of equivalence classes States within the same class have the same transience or recurrence properties (called solidarity properties ) Irreducibility means there is only one class We are now ready to consider the longterm behavior of a DTMC. 2 Where are we so far? Limiting Distributions Limiting Probabilities Dependent on the Starting State Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) The Theorem (for finite state spaces) Where we are heading next. Theorem A finite state, irreducible , aperiodic Markov chain has a limiting distribution that satisfies the steadystate equations . In fact, this distribution is the unique nonnegative solution to said equations. 3 Where are we so far? Limiting Distributions Limiting Probabilities Dependent on the Starting State Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Definition Example Outstanding Orders Stationary Distribution Limiting Distribution as a Stationary Distribution Limiting Distribution Definition Definition For a fixed states i and j if lim n p ( n ) ij exists it is called a limiting probability . If the limit exists for all j and it sums (over j) to 1, then { lim n p ( n ) ij for j X } , is called the limiting distribution . There are two natural questions When does the limiting distribution exist? When is the limiting distribution independent of the initial state? If the limiting distribution exists and is independent of the initial state...is it easy to compute? 4 Where are we so far? Limiting Distributions Limiting Probabilities Dependent on the Starting State Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Definition Example Outstanding Orders Stationary Distribution Limiting Distribution as a Stationary Distribution Outstanding Orders The people at a large logistics company have been working for the same clients for almost a century Since the operations are somewhat static, it is safe to assume...
View Full
Document
 Spring '09
 RESNIK

Click to edit the document details