# lecture-09 - Chapter 9 Markov Pro cesses This lecture...

This preview shows pages 1–3. Sign up to view the full content.

Chapter 9 Markov Processes This lecture begins our study of Markov processes. Section 9.1 is mainly “ideological”: it formally deﬁnes the Markov property for one-parameter processes, and explains why it is a nat- ural generalization of both complete determinism and complete sta- tistical independence. Section 9.2 introduces the description of Markov processes in terms of their transition probabilities and proves the existence of such processes. 9.1 The Correct Line on the Markov Property The Markov property is the independence of the future from the past, given the present. Let us be more formal. Deﬁnition 99 (Markov Property) A one-parameter process X is a Markov process with respect to a ﬁltration F when X t is adapted to the ﬁltration, and, for any s > t , X s is independent of F t given X t , X s | = F t | X t . If no ﬁltration is mentioned, it may be assumed to be the natural one generated by X . If X is also conditionally stationary, then it is a time-homogeneous (or just homogeneous ) Markov process. Lemma 100 Let X + t stand for the collection of X u , u > t . If X is Markov, then X + t F t | X t . Proof: See Exercise 9.1. ± There are two routes to the Markov property. One is the path followed by Markov himself, of desiring to weaken the assumption of strict statistical inde- pendence between variables to mere conditional independence. In fact, Markov speciﬁcally wanted to show that independence was not a necessary condition for the law of large numbers to hold, because his arch-enemy claimed that it was, 49

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
CHAPTER 9. MARKOV PROCESSES 50 and used that as grounds for believing in free will and Christianity. 1 It turns out that all the key limit theorems of probability — the weak and strong laws of large numbers, the central limit theorem, etc. — work perfectly well for Markov processes, as well as for IID variables. The other route to the Markov property begins with completely deterministic systems in physics and dynamics. The state of a deterministic dynamical system is some variable which ﬁxes the value of all present and future observables. As a consequence, the present state determines the state at all future times.
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 12/20/2011 for the course STAT 36-754 taught by Professor Schalizi during the Spring '06 term at University of Michigan.

### Page1 / 5

lecture-09 - Chapter 9 Markov Pro cesses This lecture...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online