lecture21 - Lecture 21 : Continuous Time Markov Chains STAT...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Lecture 21 : Continuous Time Markov Chains STAT 150 Spring 2006 Lecturer: Jim Pitman Scribe: Stephen Bianchi <> ( These notes also include material from the subsequent guest lecture given by Ani Adhikari. ) Consider a continuous time stochastic process ( X t , t 0) taking on values in the fnite state space S = { 0 , 1 , 2 , . . . , N } . Recall that in discrete time, given a transition matrix P = p ( i, j ) with i, j S , the n -step transition matrix is simply P n = p n ( i, j ), and that the Following relationship holds: P n P m = P n + m . Where p n ( i, j ) is the probability that the process moves From state i to state j in n transitions. That is, p n ( i, j ) = P i ( X n = j ) , p n ( i, j ) = P ( X n = j | X 0 = i ) , p n ( i, j ) = P ( X m + n = j | X m = i ) . Now moving to continuous time, we say that the process ( X t , t 0) is a continuous time markov chain iF the Following properties hold For all i, j S , t, s 0: P t ( i, j ) 0, N j =0 P t ( i, j ) = 1, P ( X t + s = j | X 0 = i ) = N k =0 P ( X t + s = j | X s = k ) P ( X s = k | X 0 = i ), or P ( X t + s = j | X 0 = i ) = N k =0 P s ( i, k ) P t ( k, j ). This last property can be written in matrix Form as P s + t = P s P t . This is known as the Chapman-Kolmogorov equation (the semi-group property). Example 21.1 (Poissonize a Discrete Time Markov Chain) Consider a pois- son process ( N t , t 0) with rate λ , and a jump chain with transition matrix ˆ P , which 21-1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Lecture 21: Continuous Time Markov Chains 21-2 makes jumps at times ( N t , t 0) . Let X t = Y N t ( Y N t is the value of the jump chain at time N t ), where X t takes values in the discrete state space S . Assume ( Y 0 , Y 1 , . . . ) and N t are independent. Find P t for the process X t . By de±nition P t ( i, j ) = P ( X t = j | X 0 = i ) . Condition on N t to give, P t ( i, j ) = X n =0 P ( N t = n ) ˆ P n ( i, j ) = X n =0 e - λt ( λt ) n n ! ˆ P n ( i, j ) = e - λt X n =0 ( λt ˆ P ) n ( i, j ) n ! = e
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 8

lecture21 - Lecture 21 : Continuous Time Markov Chains STAT...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online