Lecture9

# 23 vij pij 0 1 i fl i absorbing class transient class

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: tates (0.1, 2) Absorbing State (3) Markov Chains Classifications Classifications (cont’d) Periodic Markov Chain Transient Class l States may only be visited periodically 23) 6‘ 1 0 1 1 2 1 Transient States (0, 1, 2) (period=3 epochs) Recurrent Sxes (3,4) Classifications l Definition Aperiodic Markov Chain Ergodk Markov Chain All states may be visited at any epoch A markov chain is called Ergodic if l lt is Irreducible - its states are all in one class l All states are Aperiodic l All states are Recurrent Atter a (large) number of transitions, an Ergodic markov chain will eventually reach a steady state 35-1 ’ CT? 2 1 .- EM-602 / QM-710 (NJ) Lecture 9 Page 9-6 Steady State l l l Steady State Long-Run Probability Long-Run Probability We can predict the long run behavior of a system described as an Ergodic markov chain As more transitions occur, the system becomes less dependent on initial conditions State of system will eventually be a function only of the transition probabilities Given an ergodk markov chain pfl) pcnj (where n is very large) becomes a matrix with identkai rows A stationary distribution. long-run, Or steady-state vector can be found x - A(“), n+ao These nth step (n+oo) absolute probabilities are unaffected by the initial state vector Ato) l l l Steady-State Vector Steady State Vector Independent of I.C. (proof) Computations A’“’ = Calculating high powers of PC”) can be computa~ionally intensive - instead . . . Consider a system at equilibrium in x = A(“), n-r-: which The steady state vector is given by x The one-step transition matrix is P One more transition at equilibrium results in vector A@*lb A@)P n+c This is equivalent to x = x P /pp(“) 1.4286 57141 “” = [I ‘1,J286 l 5714 1 l A”’ = [ A286 /I”’ = [0 l] A”’ = [ .S 5714 l A”’ = [A286 5714 l .5] A”’ = [ A286 .5714] l .. Example l l l l l...
View Full Document

## This document was uploaded on 03/31/2014 for the course MS 602 at NJIT.

Ask a homework question - tutors are online