# F x x n m1 x note that theorems 121 and 123 do not

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: in each state.” Theorem 1.21. Asymptotic frequency. Suppose I and R. If Nn (y ) be the number of visits to y up to time n, then Nn (y ) 1 ! n Ey T y We will see later that we may have Ey Ty = 1 in which case the limit is 0. As a corollary we get the following. Theorem 1.22. If I and S hold, then ⇡ (y ) = 1/Ey Ty and hence the stationary distribution is unique. In the next two examples we will be interested in the long run cost associated with a Markov chain. For this, we will need the following extension of Theorem 1.21. (Take f (x) = 1 if x = y and 0 otherwise to recover the previous result.) P Theorem 1.23. Suppose I , S , and x |f (x)|⇡ (x) &lt; 1 then n X 1X f (Xm ) ! f (x)⇡ (x) n m=1 x Note that Theorems 1.21 and 1.23 do not require aperiodicity. To illustrate the use of Theorem 1.23, we consider 27 1.5. LIMIT BEHAVIOR Example 1.23. Repair chain (continuation of 1.7). A machine has three critical parts that are subject to failure, but can function as long as two of these parts a...
View Full Document

Ask a homework question - tutors are online