mc1 - Winter 2010 Pstat160a Hand-out MC#3 Applications of...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Winter 2010 Pstat160a Hand-out MC#3 Applications of making a state absorbing 1. General Remarks If a Markov Chain Y n has an absorbing state a and transition matrix P y whose n th power is P n y , then P n y ( i,a ) represent not only the usual n -steps probability, but also the probability that Y n reached a in n or less steps , since the chain “get stuck” in a . Similarly, P n y ( i,j ) , i,j 6 = a represent the probability that Y n never visited a in the first N steps. Now suppose that the chain X n has the same transition probability than Y n , except that a is not absorbing. The chains X n and Y n behave the same way until they reach the state a , that is their transition probabilities are identical up to that point. 2. Visit to a state in less than n steps Starting with a Markov chain with matrix P , suppose we want to find the probability that the chain visit a particular state in n steps or less. It should be clear by taking complement that this will give us also the probability that the chain never visit that state in the first n steps. The basic idea is to make this state absorbing (call it a ) and leave the rest of the chain intact. The resulting new chain
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 3

mc1 - Winter 2010 Pstat160a Hand-out MC#3 Applications of...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online