Stochastic - i Essentials of Stochastic Processes Rick...

i
Essentials of Stochastic Processes
Rick Durrett
30
40
50
60
70
10 Ͳ Sep
10 Ͳ Jun
10 Ͳ May
at expiry
0
10
20
500 520 540 560 580 600 620 640 660 680 700
Almost Final Version of the 2nd Edition, December, 2011
Copyright 2011, All rights reserved.

ii
Preface
Between the first undergraduate course in probability and the first graduate
course that uses measure theory, there are a number of courses that teach
Stochastic Processes to students with many di ↵ erent interests and with varying
degrees of mathematical sophistication. To allow readers (and instructors) to
choose their own level of detail, many of the proofs begin with a nonrigorous
answer to the question “ Why is this true? ” followed by a Proof that fills in
the missing details. As it is possible to drive a car without knowing about the
working of the internal combustion engine, it is also possible to apply the theory
of Markov chains without knowing the details of the proofs. It is my personal
philosophy that probability theory was developed to solve problems, so most of
our e ↵ ort will be spent on analyzing examples. Readers who want to master the
subject will have to do more than a few of the twenty dozen carefully chosen
exercises.
This book began as notes I typed in the spring of 1997 as I was teaching
ORIE 361 at Cornell for the second time. In Spring 2009, the mathematics
department there introduced its own version of this course, MATH 474. This
started me on the task of preparing the second edition. The plan was to have
this finished in Spring 2010 after the second time I taught the course, but when
May rolled around completing the book lost out to getting ready to move to
Durham after 25 years in Ithaca. In the Fall of 2011, I taught Duke’s version
of the course, Math 216, to 20 undergrads and 12 graduate students and over
the Christmas break the second edition was completed.
The second edition di ↵ ers substantially from the first, though curiously the
length and the number of problems has remained roughly constant. Throughout
the book there are many new examples and problems, with solutions that use
the TI-83 to eliminate the tedious details of solving linear equations by hand.
My students tell me I should just use MATLAB and maybe I will for the next
edition.
The Markov chains chapter has been reorganized. The chapter on Poisson
processes has moved up from third to second, and is now followed by a treatment
of the closely related topic of renewal theory. Continuous time Markov chains
remain fourth, with a new section on exit distributions and hitting times, and
reduced coverage of queueing networks. Martingales, a di ffi cult subject for
students at this level, now comes fifth, in order to set the stage for their use in
a new sixth chapter on mathematical finance. The treatment of finance expands
the two sections of the previous treatment to include American options and the
the capital asset pricing model. Brownian motion makes a cameo appearance
in the discussion of the Black-Scholes theorem, but in contrast to the previous
edition, is not discussed in detail.