lecture-09 - Chapter 9 Markov Pro cesses This lecture...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Chapter 9 Markov Processes This lecture begins our study of Markov processes. Section 9.1 is mainly “ideological”: it formally defines the Markov property for one-parameter processes, and explains why it is a nat- ural generalization of both complete determinism and complete sta- tistical independence. Section 9.2 introduces the description of Markov processes in terms of their transition probabilities and proves the existence of such processes. 9.1 The Correct Line on the Markov Property The Markov property is the independence of the future from the past, given the present. Let us be more formal. Definition 99 (Markov Property) A one-parameter process X is a Markov process with respect to a filtration F when X t is adapted to the filtration, and, for any s > t , X s is independent of F t given X t , X s | = F t | X t . If no filtration is mentioned, it may be assumed to be the natural one generated by X . If X is also conditionally stationary, then it is a time-homogeneous (or just homogeneous ) Markov process. Lemma 100 Let X + t stand for the collection of X u , u > t . If X is Markov, then X + t F t | X t . Proof: See Exercise 9.1. ± There are two routes to the Markov property. One is the path followed by Markov himself, of desiring to weaken the assumption of strict statistical inde- pendence between variables to mere conditional independence. In fact, Markov specifically wanted to show that independence was not a necessary condition for the law of large numbers to hold, because his arch-enemy claimed that it was, 49
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
CHAPTER 9. MARKOV PROCESSES 50 and used that as grounds for believing in free will and Christianity. 1 It turns out that all the key limit theorems of probability — the weak and strong laws of large numbers, the central limit theorem, etc. — work perfectly well for Markov processes, as well as for IID variables. The other route to the Markov property begins with completely deterministic systems in physics and dynamics. The state of a deterministic dynamical system is some variable which fixes the value of all present and future observables. As a consequence, the present state determines the state at all future times.
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 12/20/2011 for the course STAT 36-754 taught by Professor Schalizi during the Spring '06 term at University of Michigan.

Page1 / 5

lecture-09 - Chapter 9 Markov Pro cesses This lecture...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online