Stochastic

Px va vb which gives the desired result 46 chapter 1

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: bing states and let r(x, y ) the restriction of p to x, y 2 C (i.e., the 3 ⇥ 3 matrix inside the black lines in the transition probability). In this notation then the matrix above is I r. Solving gives 0 1 01 0 1 h(1) .6 .8769 @ h(0) A = (I r) 1 @ 0 A = @.6923A h( 1) 0 .4154 General solution. Suppose that the server wins each point with probability w. If the game is tied then after two points, the server will have won with probability w2 , lost with probability (1 w)2 , and returned to a tied game with probability 2w(1 w), so h(0) = w2 + 2w(1 w)h(0). Since 1 2w(1 w) = w2 + (1 w)2 , solving gives h(0) = w2 w2 + (1 w)2 Figure 1.3 graphs this function. Having worked two examples, it is time to show that we have computed the right answer. In some cases we will want to guess and verify the answer. In those situations it is nice to know that the solution is unique. The next result proves this. Theorem 1.27. Consider a Markov chain with finite state space S . Let a and b be two points in S , and let C = S {a, b}. Suppose h(a) = 1, h(b) = 0, and that for x 2 C we have X h(x) = p(x, y )h(y ) (1.16) y If Px (Va ^ Vb < 1) >...
View Full Document

Ask a homework question - tutors are online