{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Bruce Hajek Random Processes Notes

Bruce Hajek Random Processes Notes - Contents 1 Getting...

Info icon This preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
Contents 1 Getting Started 1 1.1 The axioms of probability theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Independence and conditional probability . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Random variables and their distribution . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4 Functions of a random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.5 Expectation of a random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.6 Frequently used distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.7 Jointly distributed random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 1.8 Cross moments of random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 1.9 Conditional densities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 1.10 Transformation of random vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 1.11 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2 Convergence of a Sequence of Random Variables 41 2.1 Four definitions of convergence of random variables . . . . . . . . . . . . . . . . . . . 41 2.2 Cauchy criteria for convergence of random variables . . . . . . . . . . . . . . . . . . 52 2.3 Limit theorems for sequences of independent random variables . . . . . . . . . . . . 56 2.4 Convex functions and Jensen’s inequality . . . . . . . . . . . . . . . . . . . . . . . . 59 2.5 Chernoff bound and large deviations theory . . . . . . . . . . . . . . . . . . . . . . . 60 2.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3 Random Vectors and Minimum Mean Squared Error Estimation 73 3.1 Basic definitions and properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3.2 The orthogonality principle for minimum mean square error estimation . . . . . . . . 75 3.3 Conditional expectation and linear estimators . . . . . . . . . . . . . . . . . . . . . . 80 3.3.1 Conditional expectation as a projection . . . . . . . . . . . . . . . . . . . . . 80 3.3.2 Linear estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 3.3.3 Discussion of the estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.4 Joint Gaussian distribution and Gaussian random vectors . . . . . . . . . . . . . . . 85 3.5 Linear Innovations Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.6 Discrete-time Kalman filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 3.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 iii
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
iv CONTENTS 4 Random Processes 105 4.1 Definition of a random process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 4.2 Random walks and gambler’s ruin . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 4.3 Processes with independent increments and martingales . . . . . . . . . . . . . . . . 110 4.4 Brownian motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 4.5 Counting processes and the Poisson process . . . . . . . . . . . . . . . . . . . . . . . 113 4.6 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.7 Joint properties of random processes . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.8 Conditional independence and Markov processes . . . . . . . . . . . . . . . . . . . . 119 4.9 Discrete-state Markov processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4.10 Space-time structure of discrete-state Markov processes . . . . . . . . . . . . . . . . 128 4.11 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 5 Inference for Markov Models 143 5.1 A bit of estimation theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 5.2 The expectation-maximization (EM) algorithm . . . . . . . . . . . . . . . . . . . . . 149 5.3 Hidden Markov models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 5.3.1 Posterior state probabilities and the forward-backward algorithm . . . . . . . 155 5.3.2 Most likely state sequence – Viterbi algorithm . . . . . . . . . . . . . . . . . 158 5.3.3 The Baum-Welch algorithm, or EM algorithm for HMM . . . . . . . . . . . . 159 5.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 5.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 6 Dynamics of Countable-State Markov Models 165 6.1 Examples with finite state space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 6.2 Classification and convergence of discrete-time Markov processes . . . . . . . . . . . 167 6.3 Classification and convergence of continuous-time Markov processes . . . . . . . . . 170 6.4 Classification of birth-death processes . . . . . . . . . . . . . . . . . . . . . . . . . . 173 6.5 Time averages vs. statistical averages . . . . . . . . . . . . . . . . . . . . . . . . . . 175 6.6 Queueing systems, M/M/1 queue and Little’s law . . . . . . . . . . . . . . . . . . . . 177 6.7 Mean arrival rate, distributions seen by arrivals, and PASTA . . . . . . . . . . . . . 180 6.8 More examples of queueing systems modeled as Markov birth-death processes . . . 182 6.9 Foster-Lyapunov stability criterion and moment bounds . . . . . . . . . . . . . . . . 184 6.9.1 Stability criteria for discrete-time processes . . . . . . . . . . . . . . . . . . . 184 6.9.2 Stability criteria for continuous time processes . . . . . . . . . . . . . . . . . 192 6.10 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 7 Basic Calculus of Random Processes 205 7.1 Continuity of random processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 7.2 Mean square differentiation of random processes . . . . . . . . . . . . . . . . . . . . 211 7.3 Integration of random processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 7.4 Ergodicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 7.5 Complexification, Part I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
Image of page 2
CONTENTS v 7.6 The Karhunen-Lo` eve expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 7.7 Periodic WSS random processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 7.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 8
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern