MIT6_262S11_lec20

J if we truncate this process to k states then pj j pj

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ✒✑ ③ 1.6 2 ② ✒✑ ③ 3 ... ✖✕ 3.2 Same process in terms of {qij } Using pj qj,j +1 = pj +1qj +1,j , we see that pj +1 = 3 pj , so 4 ￿ j and pj = (1/4) (3/4) pj νj = ∞. j If we truncate this process to k states, then pj ￿ j pj νj ￿ ￿ ￿k ￿ ￿ ￿j ￿ ￿k ￿ ￿ ￿k−j 3 3 1 2 2 = 1− ; πj = 1− 4 4 3 3 3 ￿ ￿ ￿ ￿k ￿ ￿￿ ￿k 1 3 3 = 1− − 1 → ∞ 2 4 2 1 4 ￿ 4 Reversibility for Markov processes For any Markov chain in steady state, the backward transition probabilities Pi∗ are defined as j ∗ πiPij = πj Pji There is nothing mysterious here, just ￿ ￿ Pr Xn = j, Xn+1 = i ￿ ￿ ￿ ￿ = Pr Xn+1 = i Pr Xn=j |Xn+1=i ￿ ￿ = Pr {Xn = j } Pr Xn+1=i|Xn=j This also holds for the embedded chain of a Markov process. ✛ State i ✲✛ State j , rate νj t1 ✲✛ State k ✲ t2 5 ✛ State i ✲✛ t1 State j , rate νj ✲✛ State k ✲ t2 Moving right, after entering state j , the exit rate is νj , i.e., we exit in each δ with probability νj δ . The same holds moving left. That is, a Poisson process is clearly reversible from the incremental definition. Thus {πi} and {νi} are the same going left as going right 6 Note that the probability of having a (right) transition ∗ from state j to k in (t, t+δ ) is pj qjk δ . Similarly, if qkj is the left-going process transition rate, the probability ∗ of having the same transition is pk qkj . Thus ∗ pj qjk = pk qkj ∗ ∗ By fiddling equations, qkj = νk Pkj . ∗ Def: A MP is reversible if qij = qij for all i, j ￿ Assuming p ositive recurrence and i πi/νi < ∞, the MP process is revers...
View Full Document

Ask a homework question - tutors are online