# Stochastic

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ter the matrix into the computer as say [A]. The selecting the NAMES we can enter [A] ^ 2 on the computation line to get A2 . If we use this procedure to compute A20 we get a matrix with three rows that agree in the ﬁrst six decimal places with .468085 .340425 .191489 Later we will see that as n ! 1, pn converges to a matrix with all three rows equal to (22/47, 16/47, 9/47). To explain our interest in pm we will now prove: Theorem 1.1. The m step transition probability P (Xn+m = j |Xn = i) is the mth power of the transition matrix p. The key ingredient in proving this is the Chapman–Kolmogorov equation X pm+n (i, j ) = pm (i, k ) pn (k, j ) (1.2) k Once this is proved, Theorem 1.1 follows, since taking n = 1 in (1.2), we see that X pm+1 (i, j ) = pm (i, k ) p(k, j ) k That is, the m +1 step transition probability is the m step transition probability times p. Theorem 1.1 now follows. Why is (1.2) true? To go from i to j in m + n steps, we have to go from i to some state k in m steps and then from k to j in n steps. The Markov property imp...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online