Unformatted text preview: bing states and let r(x, y ) the restriction
of p to x, y 2 C (i.e., the 3 ⇥ 3 matrix inside the black lines in the transition
probability). In this notation then the matrix above is I r. Solving gives
0
1
01 0
1
h(1)
.6
.8769
@ h(0) A = (I r) 1 @ 0 A = @.6923A
h( 1)
0
.4154
General solution. Suppose that the server wins each point with probability w. If the game is tied then after two points, the server will have
won with probability w2 , lost with probability (1 w)2 , and returned to a
tied game with probability 2w(1 w), so h(0) = w2 + 2w(1 w)h(0). Since
1 2w(1 w) = w2 + (1 w)2 , solving gives
h(0) = w2 w2
+ (1 w)2 Figure 1.3 graphs this function.
Having worked two examples, it is time to show that we have computed the
right answer. In some cases we will want to guess and verify the answer. In
those situations it is nice to know that the solution is unique. The next result
proves this.
Theorem 1.27. Consider a Markov chain with ﬁnite state space S . Let a and
b be two points in S , and let C = S {a, b}. Suppose h(a) = 1, h(b) = 0, and
that for x 2 C we have
X
h(x) =
p(x, y )h(y )
(1.16)
y If Px (Va ^ Vb < 1) >...
View
Full
Document
 Spring '10
 DURRETT
 The Land

Click to edit the document details