STA3007_0506_t07

STA3007_0506_t07 - STA 3007 Applied Probability Tutorial 7...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
STA 3007 Applied Probability 2005 Tutorial 7 1. Revision (a) Review on Probability (b) Introduction to Markov Chain (c) First Step Analysis i. Probability of absorption in k, given an initial state i U ik = P { X T = k | X 0 = i } ii. Expected absorption time given an inital transient state i v i = E [ T | X 0 = i ] , i = 0 , 1 , 2 ,...,r - 1 iii. For a transition matrix P = ± Q R 0 I ² 0 is an (N-r+1) by r matrix with all entries being zero. I is an (N-r+1) by (N-r+1) identity matrix. Q = [ Q ij ] where Q ij = P ij . Then U = [ U r U r + 1 ... U N ] = R + QU = ( I - Q ) - 1 R , where U k = ( U 0k , U 1k ,..., U r - 1 , k ) 0 and V = ( v 0 , v 1 ,..., v r - 1 ) 0 = [ 1 ] + QV = ( I - Q ) - 1 [ 1 ] (d) The Long Run Behavior of Markov Chains i. Regular Transition Probability Matrices ii. Limiting Probability Distribution For a Markov Matrix P ij , the limiting distribution is defined as lim n →∞ P ( n ) ij = π j > 0 , for j = 0 , 1 , ··· ,N Let P be a Regular Markov matrix on the states 0 , 1 , ··· ,N
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 05/21/2011 for the course STA 3007 taught by Professor Kb during the Spring '11 term at CUHK.

Page1 / 4

STA3007_0506_t07 - STA 3007 Applied Probability Tutorial 7...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online