3 classication of states we begin with some important

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: lassification of States We begin with some important notation. We are often interested in the behavior of the chain for a fixed initial state, so we will introduce the shorthand Px (A) = P (A|X0 = x) Later we will have to consider expected values for this probability and we will denote them by Ex . Let Ty = min{n 1 : Xn = y } be the time of the first return to y (i.e., being there at time 0 doesn’t count), and let ⇢yy = Py (Ty < 1) be the probability Xn returns to y when it starts at y . Note that if we didn’t exclude n = 0 this probability would always be 1. Intuitively, the Markov property implies that the probability Xn will return to y at least twice is ⇢2 , since after the first return, the chain is at y , and the yy probability of a second return following the first is again ⇢yy . To show that the reasoning in the last paragraph is valid, we have to introduce a definition and state a theorem. We say that T is a stopping time if the occurrence (or nonoccurrence) of the event “we stop at time n,” {T = n}, can be determined by looking at the values o...
View Full Document

Ask a homework question - tutors are online