Continuous time markov chains formal construction

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: In continuous time, as in discrete time, the transition probability satisfies Theorem 4.1. Chapman–Kolmogorov equation. X ps (i, k )pt (k, j ) = ps+t (i, j ) k Why is this true? In order for the chain to go from i to j in time s + t, it must be in some state k at time s, and the Markov property implies that the two parts of the journey are independent. Proof. Breaking things down according to the state at time s, we have X P (Xs+t = j |X0 = i) = P (Xs+t = j, Xs = k |X0 = i) k Using the definition of conditional probability and the Markov property, the above is X X = P (Xs+t = j |Xs = k, X0 = i)P (Xs = k |X0 = i) = pt (k, j )ps (i, k ) k k (4.1) shows that if we know the transition probability for t < t0 for any t0 > 0, we know it for all t. This observation and a large leap of faith (which we will justify later) suggests that the transition probabilities pt can be determined from their derivatives at 0: q (i, j ) = lim h!0 ph (i, j ) h for j 6= i (4.1) If this limit exists (and it will in all the cases we consider) we will call q (i, j ) the jump rate from i to j . To explain this name we will compute the: Jump rates for Example 4.1. The probability of at least t...
View Full Document

Ask a homework question - tutors are online