Stochastic

Let vy minn 0 xn y be the time of the rst visit to

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: x (y )/Ex Tx is a stationary distribution. Since µx (x) = 1 we see that ⇡ (x) = 1 Ex T x If there are k states then the stationary distribution ⇡ can be computed by the following procedure. Form a matrix A by taking the first k 1 columns of p I and adding a final column of 1’s. The equations ⇡ p = ⇡ and ⇡1 + · · · ⇡k = 1 are equivalent to ⇡A = 0 . . . 0 1 so we have ⇡= 0 ... 0 1A 1 or ⇡ is the bottom row of A 1 . In two situations, the stationary distribution is easy to compute. (i) If P the chain is doubly stochastic, i.e., x p(x, y ) = 1, and has k states, then the stationary distribution is uniform ⇡ (x) = 1/k . (ii) ⇡ is a stationary distribution if the detailed balance condition holds ⇡ (x)p(x, y ) = ⇡ (y )p(y, x) Birth and death chains, defined by the condition that p(x, y ) = 0 if |x y | > 1 always have stationary distributions with this property. If the state space is `, ` + 1, . . . r then ⇡ can be found by setting ⇡ (`) = c, solving for ⇡ (x) for ` < x r, and then choosing c to make the probabilities sum to 1. Convergence theorems Transient states y have pn (x, y ) !...
View Full Document

This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online