Stochastic

# Writing f x for the density function of tq on 0 1 we

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 5 0 0 1 2 3 1 0 0 0 2/5 0 3/5 0 2/5 0 1 0 Let g (i) = Ei V0 . g (0) = 0. Taking into account the rates at which jumps occur we have 1 2 /5 + (2) 5 g 13 2/5 g (2) = + g (1) + (3) 55 g 1 g (3) = + g (2) 3 g (1) = Inserting the last equation in the second one: g (2) = 13 2 2 + g (1) + + g (2) 55 15 5 or (3/5)g (2) = (1/3) + (3/5)g (1). Multiplying by 2/3’s and inserting this in the ﬁrst equation we have 122 g (1) = + + g (1) 595 so (3/5)g (1) = 19/45 and g (1) = 19/27. To develop the analogue of (4.19) for exit times we note that if g (i) = Ei VA then g (i) = 0 for i 2 A, and for i 62 A g (i) = 1 i Multiplying each side by + X q (i, j ) g (j ) i j 6=i Q(i, i) we have X Q(i, i)g (i) = 1 + Q(i, j )g (j ) i = j 6=i which simpliﬁes to X j Q(i, j )g (j ) = 1 for i 62 A. 136 CHAPTER 4. CONTINUOUS TIME MARKOV CHAINS Writing 1 for a vector of 1’s the solution, and writing R for the part of Q with i, j, 2 Ac : g = R 11 (4.21) In the barbershop example, the matrix R is 1 2 3 which has R 1 1 5 3 0 0 1/3 = @1/3 1/3 2 2 5 3 2/9 5/9 5/9 3 0 2 3 1 4/27 10/27A 19/27 Adding the entries on the ﬁrst row we ﬁnd g (1) = 1/...
View Full Document

## This document was uploaded on 03/06/2014 for the course MATH 4740 at Cornell.

Ask a homework question - tutors are online