Unformatted text preview: ✒✑ ③ 1.6 2 ②
✒✑ ③ 3 ... ✖✕ 3.2 Same process in terms of {qij } Using pj qj,j +1 = pj +1qj +1,j , we see that pj +1 = 3 pj , so
4
j and
pj = (1/4) (3/4)
pj νj = ∞.
j If we truncate this process to k states, then
pj
j pj νj
k j
k k−j
3
3
1
2
2
=
1−
;
πj =
1−
4
4
3
3
3
k k
1
3
3
= 1−
− 1 → ∞ 2
4
2
1
4 4 Reversibility for Markov processes
For any Markov chain in steady state, the backward
transition probabilities Pi∗ are deﬁned as
j
∗
πiPij = πj Pji
There is nothing mysterious here, just Pr Xn = j, Xn+1 = i = Pr Xn+1 = i Pr Xn=j Xn+1=i
= Pr {Xn = j } Pr Xn+1=iXn=j
This also holds for the embedded chain of a Markov
process. ✛ State i ✲✛ State j , rate νj t1 ✲✛ State k ✲ t2
5 ✛ State i ✲✛ t1 State j , rate νj ✲✛ State k ✲ t2 Moving right, after entering state j , the exit rate is νj ,
i.e., we exit in each δ with probability νj δ . The same
holds moving left.
That is, a Poisson process is clearly reversible from
the incremental deﬁnition.
Thus {πi} and {νi} are the same going left as going
right 6 Note that the probability of having a (right) transition
∗
from state j to k in (t, t+δ ) is pj qjk δ . Similarly, if qkj is
the leftgoing process transition rate, the probability
∗
of having the same transition is pk qkj . Thus
∗
pj qjk = pk qkj
∗
∗
By ﬁddling equations, qkj = νk Pkj .
∗
Def: A MP is reversible if qij = qij for all i, j Assuming p ositive recurrence and i πi/νi < ∞, the MP
process is revers...
View
Full
Document
This note was uploaded on 01/13/2012 for the course ELECTRICAL 6.262 taught by Professor Staff during the Fall '11 term at MIT.
 Fall '11
 staff

Click to edit the document details