Markov_chainsIII-beamer (1)

# Markov_chainsIII-beamer (1) - Where are we so far Limiting...

This preview shows pages 1–6. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Where are we so far? Limiting Distributions Limiting Probabilities Dependent on the Starting State Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Introductory Engineering Stochastic Processes, ORIE 3510 Instructor: Professor Mark E. Lewis School of Operations Research and Information Engineering Cornell University Disclaimer : Notes are only meant as a lecture supplement not substitute! 1 Where are we so far? Limiting Distributions Limiting Probabilities Dependent on the Starting State Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Where are we so far? States that we visit infinitely many times (given that we start in said state) are called recurrent . Otherwise they are called transient . Accessibility leads to communication and the creation of equivalence classes States within the same class have the same transience or recurrence properties (called solidarity properties ) Irreducibility means there is only one class We are now ready to consider the long-term behavior of a DTMC. 2 Where are we so far? Limiting Distributions Limiting Probabilities Dependent on the Starting State Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) The Theorem (for finite state spaces) Where we are heading next. Theorem A finite state, irreducible , aperiodic Markov chain has a limiting distribution that satisfies the steady-state equations . In fact, this distribution is the unique non-negative solution to said equations. 3 Where are we so far? Limiting Distributions Limiting Probabilities Dependent on the Starting State Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Definition Example – Outstanding Orders Stationary Distribution Limiting Distribution as a Stationary Distribution Limiting Distribution – Definition Definition For a fixed states i and j if lim n →∞ p ( n ) ij exists it is called a limiting probability . If the limit exists for all j and it sums (over j) to 1, then { lim n →∞ p ( n ) ij for j ∈ X } , is called the limiting distribution . There are two natural questions When does the limiting distribution exist? When is the limiting distribution independent of the initial state? If the limiting distribution exists and is independent of the initial state...is it easy to compute? 4 Where are we so far? Limiting Distributions Limiting Probabilities Dependent on the Starting State Periodicity Limiting Probability v. Limiting Distributions The Steady State Equations (finite or infinite state spaces) Definition Example – Outstanding Orders Stationary Distribution Limiting Distribution as a Stationary Distribution Outstanding Orders The people at a large logistics company have been working for the same clients for almost a century Since the operations are somewhat static, it is safe to assume...
View Full Document

{[ snackBarMessage ]}

### Page1 / 30

Markov_chainsIII-beamer (1) - Where are we so far Limiting...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online