Unformatted text preview: ctor)  State of the system in epoch
n, given as vector [email protected]=[ a,(* 4j* 4,(* ]
Initial State Vector  If n=0, then nlo)
describes the initial conditions of the
system l The absolute probability for state n can
be computed according to the following: Matrix arithmetic can be used to find the
3rd step absolute probabilities: l A”’ = [~;‘)17;J)...l?:))]
= ,#O)p(J) = /fl’p’Z’ = /fZ)fHl) nstep Matrices
l l Example ChapmanKolomogorov Eq’s nstep Probabilities The ChapmanKolomorov Equations
show that successive states can be
described according to: l Find P(2), Glven the One Step Matrix v”) p = p(” =[I: ;] The equivalent matrix relationship is as
follows: p(n) = penMp(N r.52 p(z) = AS1 1.36 .6I] EM602 I QM710 (NJ) Lecture 9
Page 94 Example (cont’d) Example (cont’d) nstep Probabilities nstep Probabilities Find PCS l PC’) # ::I pm It is interesting to observe that as the
number of transitions gets larger, the
rows become more similar l =[z :;I (*u) J4286 57141
P1.4286 57141 .60f31 po, =r.392 1.456 5441 Example Example (cont’d) nth State Vectors nth State Vectors Flnd A(z), Given the Initial Condition Ato) l A'O' = [ .7 .3] l r52 A81
PC*) =1.X 6IJ
. Find A@) Ii@) =[.7 31 r392 P')=~e4s r52 .rsl
#' = ‘4’“‘P’z’ =[ .7 .3][_x &J A"' = A'"'PJ' =[.7 A'*' = [A72 .528] .6061
WI r.392 .6081
.156 jerj
3 1 A"' =[A112 _5888] Markov Chains
l l Markov Chains fenninology Classifications Markov Chain  A combination of OneStep Transition matrix and an Initial
State Vector
nstep Matrix  A stochastic matrix
showing probability of transitions over
n epochs l Recurrent state  A state in which revisiting is certain Pii = 1
l put”) < 1
l n<oc Absorbing state  A state from which
there is no escape piiw = 1 EM602 / QM710 (NJ) Lecture 9
Page 95 nla, Transient state  A state from which the
system can exit and Possible not return Markov Chains Markov Chains Classitkations
l l l Classitkations Transient class  a group of transient
states
Absorbing class  a group of recurrent
states from which there is no escape
Irreducible  A markov chain in which
ail states are in one class (all states are
reachable from all other states)
ViJ
or, Piif* I n 4.23 ViJ
Pij” > 0 1 I fl I Absorbing Class Transient Class j(3, cc Transient S...
View
Full
Document
This document was uploaded on 03/31/2014 for the course MS 602 at NJIT.
 Spring '94
 DonaldC.Johnson

Click to edit the document details