Unformatted text preview: Solution Gambler Problem Gambler Problem Two piayers have $3 each
In each game, each player bets $1 and
the winner takes It all
Player A Is better than 6 and has a 60%
chance of winning
The match ends when either player has
won all the money (the other is broke)
Develop a transition matrix and
classify ail states and classes l l l l Define the epoch as one game
Detlne the states of the system as the
amount of money that Player A has
Valid states are 0 (player A is broke)
through 6 (Player A has all the money)
The transition probabilities will be
based on the winning percentage of A EM602 I QM710 (NJ) Lecture 9
Page 97 Gambler Problem Gambler Problem Transition matrix (i) Transition matrix (2) 0 1 2 3 4 5 6 8 1 2 3 4 5 6
1 0 0 0 0 0 0
.4 0 .6 0 0 0 0 Gambler Problem Gambler Problem Transition matrix (3) Transition matrix (comptete) 1
0
0
.4 2
0
.6
0 3
0
0
.6 4
0
0
0 5
0
0
0 0 1 2
0 1 0 0
1 .4 0 .6
2 0 .J 0 6
0
0
0 Gambler Problem 4
0
0
0 5
0
0
0 6
0
0
0 0
.4
0
0 .6 0
0 .6
.4 0
0 0 0
0
.6
1 0
0
0
0 0
0
0
0 .4
0
0
0 Mean Recurrence Time Transitton matrix (complete)
0 1 2 3 4 5 6 3
0
0
.6 3
4
5
6 0
1
.4
0 Classiftcation
States 0 and 6
are Recurrent 6
Absorbing l l States 1 thru 5
are Transient Definition  The expected value of the
number of epochs between successive
visits to a state j
A relationship exists between steady
state vector xi and mean recurrence
time !kij accordtng to: p, =+
J EM602 I QM710 (NJ) Lecture 9
Page 98 Problem 18l 7 Problem 18l 7
(page 7121
Find the mean recurrence times for each state From the relationship: RP = II Problem 18l 7 Problem 1817 Solution
Fern the relationship: Rp = R
1
1
1
x,+n*+x,=x,
Eliminate one
3
2
5
1
~~,+fl~+$c,=~~~
RedundantEqn Solving Simultaneously.. .
= r27 28 $36 3 7 271
75 74 * .  =175 pj = Mean Recurrence Time = L
R/
r75 75 751
Jo =~jy u, ~~=[278 2 . 6 8 3.751 1
1
1
X,fXz+X,=X,
. . . and add the
3
I 5
Normalizing Equation: x,+x,+x,=1 Problem 1817
Interpretation
l l l From the steadystate vector n, and the
mean recurrence time &L, we conclude:
System will (In the long run) spend
36% of the time in state 1, 37% in state
2, and 27% in state 3
The mean time between revisits is 2.78
epochs for state 1, 2.68 epochs for
state 2, and for 3.75 epochs for state 3 EM602 I QM710 (NJ) Lecture 9
Page 99...
View
Full Document
 Spring '94
 DonaldC.Johnson
 Probability theory, ProbabiliTy Review

Click to edit the document details