Lecture9

# S 5714 l a a286 5714 l 5 a a286 5714 l example l l l

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Solution Gambler Problem Gambler Problem Two piayers have \$3 each In each game, each player bets \$1 and the winner takes It all Player A Is better than 6 and has a 60% chance of winning The match ends when either player has won all the money (the other is broke) Develop a transition matrix and classify ail states and classes l l l l Define the epoch as one game Detlne the states of the system as the amount of money that Player A has Valid states are 0 (player A is broke) through 6 (Player A has all the money) The transition probabilities will be based on the winning percentage of A EM-602 I QM-710 (NJ) Lecture 9 Page 9-7 Gambler Problem Gambler Problem Transition matrix (i) Transition matrix (2) 0 1 2 3 4 5 6 8 1 2 3 4 5 6 1 0 0 0 0 0 0 .4 0 .6 0 0 0 0 Gambler Problem Gambler Problem Transition matrix (3) Transition matrix (comptete) 1 0 0 .4 2 0 .6 0 3 0 0 .6 4 0 0 0 5 0 0 0 0 1 2 0 1 0 0 1 .4 0 .6 2 0 .J 0 6 0 0 0 Gambler Problem 4 0 0 0 5 0 0 0 6 0 0 0 0 .4 0 0 .6 0 0 .6 .4 0 0 0 0 0 .6 1 0 0 0 0 0 0 0 0 .4 0 0 0 Mean Recurrence Time Transitton matrix (complete) 0 1 2 3 4 5 6 3 0 0 .6 3 4 5 6 0 1 .4 0 Classiftcation States 0 and 6 are Recurrent 6 Absorbing l l States 1 thru 5 are Transient Definition - The expected value of the number of epochs between successive visits to a state j A relationship exists between steady state vector xi and mean recurrence time !kij accordtng to: p, =+ J EM-602 I QM-710 (NJ) Lecture 9 Page 9-8 Problem 18-l 7 Problem 18-l 7 (page 7121 Find the mean recurrence times for each state From the relationship: RP = II Problem 18-l 7 Problem 18-17 Solution Fern the relationship: Rp = R 1 1 1 -x,+--n*+--x,=x, Eliminate one 3 2 5 1 ~~,+fl~+\$c,=~~~ RedundantEqn Solving Simultaneously.. . = r27 28 \$36 3 7 271 75 74 * . - =175 pj = Mean Recurrence Time = L R/ r75 75 751 Jo =~jy u, ~~=[278 2 . 6 8 3.751 1 1 1 -X,f--Xz+-X,=X, . . . and add the 3 -I 5 Normalizing Equation: x,+x,+x,=1 Problem 18-17 Interpretation l l l From the steady-state vector n, and the mean recurrence time &L, we conclude: System will (In the long run) spend 36% of the time in state 1, 37% in state 2, and 27% in state 3 The mean time between re-visits is 2.78 epochs for state 1, 2.68 epochs for state 2, and for 3.75 epochs for state 3 EM-602 I QM-710 (NJ) Lecture 9 Page 9-9...
View Full Document

## This document was uploaded on 03/31/2014 for the course MS 602 at NJIT.

Ask a homework question - tutors are online