Stochastic

# Let vy minn 0 xn y be the time of the rst visit to

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: x (y )/Ex Tx is a stationary distribution. Since µx (x) = 1 we see that ⇡ (x) = 1 Ex T x If there are k states then the stationary distribution ⇡ can be computed by the following procedure. Form a matrix A by taking the ﬁrst k 1 columns of p I and adding a ﬁnal column of 1’s. The equations ⇡ p = ⇡ and ⇡1 + · · · ⇡k = 1 are equivalent to ⇡A = 0 . . . 0 1 so we have ⇡= 0 ... 0 1A 1 or ⇡ is the bottom row of A 1 . In two situations, the stationary distribution is easy to compute. (i) If P the chain is doubly stochastic, i.e., x p(x, y ) = 1, and has k states, then the stationary distribution is uniform ⇡ (x) = 1/k . (ii) ⇡ is a stationary distribution if the detailed balance condition holds ⇡ (x)p(x, y ) = ⇡ (y )p(y, x) Birth and death chains, deﬁned by the condition that p(x, y ) = 0 if |x y | > 1 always have stationary distributions with this property. If the state space is `, ` + 1, . . . r then ⇡ can be found by setting ⇡ (`) = c, solving for ⇡ (x) for ` < x r, and then choosing c to make the probabilities sum to 1. Convergence theorems Transient states y have pn (x, y ) !...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online