{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Bruce Hajek Random Processes Notes

# Bruce Hajek Random Processes Notes - Contents 1 Getting...

This preview shows pages 1–4. Sign up to view the full content.

Contents 1 Getting Started 1 1.1 The axioms of probability theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Independence and conditional probability . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Random variables and their distribution . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.4 Functions of a random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.5 Expectation of a random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.6 Frequently used distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.7 Jointly distributed random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 1.8 Cross moments of random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 1.9 Conditional densities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 1.10 Transformation of random vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 1.11 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2 Convergence of a Sequence of Random Variables 41 2.1 Four definitions of convergence of random variables . . . . . . . . . . . . . . . . . . . 41 2.2 Cauchy criteria for convergence of random variables . . . . . . . . . . . . . . . . . . 52 2.3 Limit theorems for sequences of independent random variables . . . . . . . . . . . . 56 2.4 Convex functions and Jensen’s inequality . . . . . . . . . . . . . . . . . . . . . . . . 59 2.5 Chernoff bound and large deviations theory . . . . . . . . . . . . . . . . . . . . . . . 60 2.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 3 Random Vectors and Minimum Mean Squared Error Estimation 73 3.1 Basic definitions and properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3.2 The orthogonality principle for minimum mean square error estimation . . . . . . . . 75 3.3 Conditional expectation and linear estimators . . . . . . . . . . . . . . . . . . . . . . 80 3.3.1 Conditional expectation as a projection . . . . . . . . . . . . . . . . . . . . . 80 3.3.2 Linear estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 3.3.3 Discussion of the estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.4 Joint Gaussian distribution and Gaussian random vectors . . . . . . . . . . . . . . . 85 3.5 Linear Innovations Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 3.6 Discrete-time Kalman filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 3.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 iii

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
iv CONTENTS 4 Random Processes 105 4.1 Definition of a random process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 4.2 Random walks and gambler’s ruin . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 4.3 Processes with independent increments and martingales . . . . . . . . . . . . . . . . 110 4.4 Brownian motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 4.5 Counting processes and the Poisson process . . . . . . . . . . . . . . . . . . . . . . . 113 4.6 Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.7 Joint properties of random processes . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.8 Conditional independence and Markov processes . . . . . . . . . . . . . . . . . . . . 119 4.9 Discrete-state Markov processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 4.10 Space-time structure of discrete-state Markov processes . . . . . . . . . . . . . . . . 128 4.11 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 5 Inference for Markov Models 143 5.1 A bit of estimation theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 5.2 The expectation-maximization (EM) algorithm . . . . . . . . . . . . . . . . . . . . . 149 5.3 Hidden Markov models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 5.3.1 Posterior state probabilities and the forward-backward algorithm . . . . . . . 155 5.3.2 Most likely state sequence – Viterbi algorithm . . . . . . . . . . . . . . . . . 158 5.3.3 The Baum-Welch algorithm, or EM algorithm for HMM . . . . . . . . . . . . 159 5.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 5.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 6 Dynamics of Countable-State Markov Models 165 6.1 Examples with finite state space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 6.2 Classification and convergence of discrete-time Markov processes . . . . . . . . . . . 167 6.3 Classification and convergence of continuous-time Markov processes . . . . . . . . . 170 6.4 Classification of birth-death processes . . . . . . . . . . . . . . . . . . . . . . . . . . 173 6.5 Time averages vs. statistical averages . . . . . . . . . . . . . . . . . . . . . . . . . . 175 6.6 Queueing systems, M/M/1 queue and Little’s law . . . . . . . . . . . . . . . . . . . . 177 6.7 Mean arrival rate, distributions seen by arrivals, and PASTA . . . . . . . . . . . . . 180 6.8 More examples of queueing systems modeled as Markov birth-death processes . . . 182 6.9 Foster-Lyapunov stability criterion and moment bounds . . . . . . . . . . . . . . . . 184 6.9.1 Stability criteria for discrete-time processes . . . . . . . . . . . . . . . . . . . 184 6.9.2 Stability criteria for continuous time processes . . . . . . . . . . . . . . . . . 192 6.10 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 7 Basic Calculus of Random Processes 205 7.1 Continuity of random processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 7.2 Mean square differentiation of random processes . . . . . . . . . . . . . . . . . . . . 211 7.3 Integration of random processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 7.4 Ergodicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 7.5 Complexification, Part I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
CONTENTS v 7.6 The Karhunen-Lo` eve expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 7.7 Periodic WSS random processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 7.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 8

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern