This preview shows page 1. Sign up to view the full content.
Unformatted text preview: tates (0.1, 2) Absorbing State (3) Markov Chains Classifications Classifications (cont’d) Periodic Markov Chain Transient Class l States may only be visited periodically 23) 6‘
1 0 1 1 2 1 Transient States (0, 1, 2) (period=3 epochs) Recurrent Sxes (3,4) Classifications
l Definition Aperiodic Markov Chain Ergodk Markov Chain All states may be visited at any epoch A markov chain is called Ergodic if
l lt is Irreducible  its states are all in one
class
l All states are Aperiodic
l All states are Recurrent
Atter a (large) number of transitions, an
Ergodic markov chain will eventually
reach a steady state 351 ’ CT?
2 1 . EM602 / QM710 (NJ) Lecture 9
Page 96 Steady State
l l l Steady State LongRun Probability LongRun Probability We can predict the long run behavior
of a system described as an Ergodic
markov chain
As more transitions occur, the system
becomes less dependent on initial
conditions
State of system will eventually be a
function only of the transition
probabilities Given an ergodk markov chain pfl)
pcnj (where n is very large) becomes a
matrix with identkai rows
A stationary distribution. longrun, Or
steadystate vector can be found
x  A(“), n+ao
These nth step (n+oo) absolute
probabilities are unaffected by the
initial state vector Ato)
l l l SteadyState Vector Steady State Vector Independent of I.C. (proof) Computations A’“’ = Calculating high powers of PC”) can be
computa~ionally intensive  instead . . .
Consider a system at equilibrium in
x = A(“), nr:
which
The steady state vector is given by x
The onestep transition matrix is P
One more transition at equilibrium
results in vector A@*lb A@)P n+c
This is equivalent to x = x P /pp(“) 1.4286 57141 “” = [I ‘1,J286 l 5714 1 l A”’ = [ A286
/I”’ = [0 l]
A”’ = [ .S 5714 l A”’ = [A286 5714 l .5] A”’ = [ A286 .5714] l .. Example
l l l l l...
View
Full
Document
This document was uploaded on 03/31/2014 for the course MS 602 at NJIT.
 Spring '94
 DonaldC.Johnson

Click to edit the document details