Lecture9

# State j after n epochs after the nfh transition

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ctor) - State of the system in epoch n, given as vector [email protected]=[ a,(* 4j* 4,(* ] Initial State Vector - If n=0, then nlo) describes the initial conditions of the system l The absolute probability for state n can be computed according to the following: Matrix arithmetic can be used to find the 3rd step absolute probabilities: l A”’ = [~;‘)17;J)...l?:))] = ,#O)p(J) = /fl’p’Z’ = /fZ)fHl) n-step Matrices l l Example Chapman-Kolomogorov Eq’s n-step Probabilities The Chapman-Kolomorov Equations show that successive states can be described according to: l Find P(2), Glven the One Step Matrix v”) p = p(” =[I: ;] The equivalent matrix relationship is as follows: p(n) = pen-Mp(N r.52 p(z) = AS1 1.36 .6-I] EM-602 I QM-710 (NJ) Lecture 9 Page 9-4 Example (cont’d) Example (cont’d) n-step Probabilities n-step Probabilities Find PCS l PC’) # ::I pm It is interesting to observe that as the number of transitions gets larger, the rows become more similar l =[z :;I (*u) J4286 57141 P-1.4286 57141 .60f31 po, =r.392 1.456 5441 Example Example (cont’d) nth State Vectors nth State Vectors Flnd A(z), Given the Initial Condition Ato) l A'O' = [ .7 .3] l r52 A81 PC*) =1.X 6-IJ . Find A@) Ii@) =[.7 31 r392 P')=~e4s r52 .rsl #' = ‘4’“‘P’z’ =[ .7 .3][_x &J A"' = A'"'PJ' =[.7 A'*' = [A72 .528] .6061 WI r.392 .6081 .156 jerj 3 1 A"' =[A112 _5888] Markov Chains l l Markov Chains fenninology Classifications Markov Chain - A combination of OneStep Transition matrix and an Initial State Vector n-step Matrix - A stochastic matrix showing probability of transitions over n epochs l Recurrent state - A state in which revisiting is certain Pii = 1 l put”) < 1 l n<oc Absorbing state - A state from which there is no escape piiw = 1 EM-602 / QM-710 (NJ) Lecture 9 Page 9-5 nla, Transient state - A state from which the system can exit and Possible not return Markov Chains Markov Chains Classitkations l l l Classitkations Transient class - a group of transient states Absorbing class - a group of recurrent states from which there is no escape Irreducible - A markov chain in which ail states are in one class (all states are reachable from all other states) ViJ or, Piif* -I n 4.23 ViJ Pij” > 0 1 I fl I Absorbing Class Transient Class j(3, cc Transient S...
View Full Document

## This document was uploaded on 03/31/2014 for the course MS 602 at NJIT.

Ask a homework question - tutors are online