i Essentials of Stochastic Processes Rick Durrett 30 40 50 60 70 10 Ͳ Sep 10 Ͳ Jun 10 Ͳ May at expiry 0 10 20 500 520 540 560 580 600 620 640 660 680 700 Almost Final Version of the 2nd Edition, December, 2011 Copyright 2011, All rights reserved.

ii Preface Between the first undergraduate course in probability and the first graduate course that uses measure theory, there are a number of courses that teach Stochastic Processes to students with many di ↵ erent interests and with varying degrees of mathematical sophistication. To allow readers (and instructors) to choose their own level of detail, many of the proofs begin with a nonrigorous answer to the question “ Why is this true? ” followed by a Proof that fills in the missing details. As it is possible to drive a car without knowing about the working of the internal combustion engine, it is also possible to apply the theory of Markov chains without knowing the details of the proofs. It is my personal philosophy that probability theory was developed to solve problems, so most of our e ↵ ort will be spent on analyzing examples. Readers who want to master the subject will have to do more than a few of the twenty dozen carefully chosen exercises. This book began as notes I typed in the spring of 1997 as I was teaching ORIE 361 at Cornell for the second time. In Spring 2009, the mathematics department there introduced its own version of this course, MATH 474. This started me on the task of preparing the second edition. The plan was to have this finished in Spring 2010 after the second time I taught the course, but when May rolled around completing the book lost out to getting ready to move to Durham after 25 years in Ithaca. In the Fall of 2011, I taught Duke’s version of the course, Math 216, to 20 undergrads and 12 graduate students and over the Christmas break the second edition was completed. The second edition di ↵ ers substantially from the first, though curiously the length and the number of problems has remained roughly constant. Throughout the book there are many new examples and problems, with solutions that use the TI-83 to eliminate the tedious details of solving linear equations by hand. My students tell me I should just use MATLAB and maybe I will for the next edition. The Markov chains chapter has been reorganized. The chapter on Poisson processes has moved up from third to second, and is now followed by a treatment of the closely related topic of renewal theory. Continuous time Markov chains remain fourth, with a new section on exit distributions and hitting times, and reduced coverage of queueing networks. Martingales, a di ffi cult subject for students at this level, now comes fifth, in order to set the stage for their use in a new sixth chapter on mathematical finance. The treatment of finance expands the two sections of the previous treatment to include American options and the the capital asset pricing model. Brownian motion makes a cameo appearance in the discussion of the Black-Scholes theorem, but in contrast to the previous edition, is not discussed in detail.