# mc1 - Winter 2010 Pstat160a Hand-out MC#3 Applications of...

This preview shows pages 1–2. Sign up to view the full content.

Winter 2010 Pstat160a Hand-out MC#3 Applications of making a state absorbing 1. General Remarks If a Markov Chain Y n has an absorbing state a and transition matrix P y whose n th power is P n y , then P n y ( i,a ) represent not only the usual n -steps probability, but also the probability that Y n reached a in n or less steps , since the chain “get stuck” in a . Similarly, P n y ( i,j ) , i,j 6 = a represent the probability that Y n never visited a in the ﬁrst N steps. Now suppose that the chain X n has the same transition probability than Y n , except that a is not absorbing. The chains X n and Y n behave the same way until they reach the state a , that is their transition probabilities are identical up to that point. 2. Visit to a state in less than n steps Starting with a Markov chain with matrix P , suppose we want to ﬁnd the probability that the chain visit a particular state in n steps or less. It should be clear by taking complement that this will give us also the probability that the chain never visit that state in the ﬁrst n steps. The basic idea is to make this state absorbing (call it a ) and leave the rest of the chain intact. The resulting new chain

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 3

mc1 - Winter 2010 Pstat160a Hand-out MC#3 Applications of...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online