Stochastic

# Suppose that server 1 serves at rate 4 while server 2

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: in with transition probability q (i, j ) r(i, j ) = i goes through the same sequence of states as Xt but stays in each one for on unit of time. Let VA = min{t : Xt 2 A} be the time of the ﬁrst visit to A. Then h(i) = Pi (X (VA ) = a) satisﬁes h(a) = 1, h(b) = 0 for b 2 A {a} and X h(i) = r(i, j )h(j ) for i 62 A. j The expected hitting time g (i) = Ei VA satsiﬁes g (a) = 0 for a 2 A and g (i) = 1 i + X r(i, j )g (j ) j for i 62 A. One can also work directly with the transition rates. In the ﬁrst case the equation with the same boundary conditions (h(a) = 1, h(b) = 0 for b 2 A {a} and for i 62 A) is X Q(i, j )h(j ) = 0 for i 62 A i In the second case, if if we let R be the part of the Q-matrix where i, j 62 A then g = R 11 where 1 is a column vector of all 1’s. P Stationary distributions. A stationary distribution has i ⇡ (i) = 1 and satisﬁes ⇡ pt = ⇡ for all t > 0, which is equivalent to ⇡ Q = 0. To solve these equations mechanically, we replace the last column of Q by all 1’s to deﬁne a matrix A and then ⇡ will be the last row of...
View Full Document

Ask a homework question - tutors are online