Stochastic

# E column vector with all 1s we can write this as i rg

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: gives ✓ ◆x ✓ ◆x q q Px (V0 = 1) = 1 and Px (V0 < 1) = . (1.23) p p To see that the form of the last answer makes sense, note that to get from x to 0 we must go x ! x 1 ! x2 . . . ! 1 ! 0, so Px (V0 < 1) = P1 (V0 < 1)x . 49 1.9. EXIT TIMES 1.9 Exit Times To motivate developments we begin with an example. Example 1.45. Two year college. In Example 1.39 we introduced a Markov chain with state space 1 = freshman, 2 = sophomore, G = graduate, D = dropout, and transition probability 1 2 G D 1 0.25 0 0 0 2 0.6 0.2 0 0 G 0 0.7 1 0 D 0.15 0.1 0 1 On the average how many years does a student take to graduate or drop out? Let g (x) be the expected time for a student starting in state x. g (G) = g (D) = 0. By considering what happens on one step g (1) = 1 + 0.25g (1) + 0.6g (2) g (2) = 1 + 0.2g (2) where the 1+ is due to the fact that after the jump has been made one year has elapsed. To solve for g , we note that the second equation implies g (2) = 1/0.8 = 1.25 and then the ﬁrst that...
View Full Document

Ask a homework question - tutors are online