# 17 and 4965 points per game respectively detailed

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: t-hand side represents the rate at which sand leaves j , while the left gives the rate at which sand arrives at j . Thus, ⇡ will be a stationary distribution if for each j the ﬂow of sand in to j is equal to the ﬂow out of j . More details. If ⇡ pt = ⇡ then X X X d ⇡ pt = ⇡ (i)p0 (i, j ) = ⇡ (i) pt (i, k )Q(k, j ) t dt i i k X XX = ⇡ (i)pt (i, k )Q(k, j ) = ⇡ (k )Q(k, j ) 0= k i k Conversely if ⇡ Q = 0 d dt X ! ⇡ (i)pt (i, j ) i = X ⇡ (i)p0 (i, j ) = t i = XX k X i ⇡ (i) X Q(i, k )pt (k, j ) k ⇡ (i)Q(i, k )pt (k, j ) = 0 i Since the derivative is 0, ⇡ pt is constant and must always be equal to ⇡ its value at 0. Lemma 4.2 implies that for any h &gt; 0, ph is irreducible and aperiodic, so by Theorem 1.19 lim pnh (i, j ) = ⇡ (j ). From this we get n!1 Theorem 4.4. If a continuous-time Markov chain Xt is irreducible and has a stationary distribution ⇡ , then lim pt (i, j ) = ⇡ (j ) t!1 We will now consider some examples. 129 4.3. LIMITING BEHAVIOR Example 4.10. L.A. weather chain. There are three states: 1 = sunny, 2 = smoggy, 3 = rainy. The weather stays sunny for an exponentially distributed number of days with mean 3, then bec...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell University (Engineering School).

Ask a homework question - tutors are online