MASSACHUSETTS INSTITUTE OF TECHNOLOGY
Fall 2006
Recitation 2
15.085/6.436
A non-measurable set in (0, 1]
Let stand for addition modulo 1 in (0, 1]. For example, 0.5 0.7 = 0.2,
instead of 1.2. If A (0, 1], and x is a number, then A x stands for the
set of

Fundamentals of probability.
6.436/15.085
LECTURE 22
Innite Markov chains. Continuous time Markov chains.
22.1. Introduction
In this lecture we cover variants of Markov chains, not covered in earlier lectures. We will discuss
innite state Markov chains. T

Fundamentals of probability.
6.436/15.085
LECTURE 18
Stochastic processes. Bernoulli and Poisson processes
18.1. Introduction
So far in the course we were considering random variables which were taking values as numbers
(real or complex) or vectors. But f

Advanced Stochastic Processes.
David Gamarnik
LECTURE 4
Large Deviations theory continued
Outline of Lecture
Large deviations lower bound.
Examples.
Bankruptcy problem.
4.1. Large deviations lower bound
We have established an upper bound on the probabi

Advanced Stochastic Processes.
David Gamarnik
LECTURE 10
Martingales and stopping times
Lecture outline
Martingales and properties.
Stopping times. Stopping theorems.
10.1. Martingales
We continue with studying examples of martingales.
Brownian motion.

Fundamentals of probability.
6.436/15.085
LECTURE 7
Covariance and correlation. Exclusion/inclusion
principle. Conditional expectations
7.1. Covariance and correlation
Variance Var(X) of a r.v. measures in some sense how variable is the r.v. Namely, how w

Fundamentals of probability.
6.436/15.085
LECTURE 12
Abstract (Lebesgue) integration.
12.1. Introduction
Recall that we have dened expected values for two kinds of r.v. discrete and continuous. If we
have a mixture of r.v. then it is not hard to guess wha

Fundamentals of probability.
6.436/15.085
LECTURE 9
Continuous random variables II. Conditional
distributions.
9.1. Joint distribution. Review
Recall, X, Y are dened to be jointly continuous if f 0 such that
x
y
P(X x, Y y) =
f (u, v)dudv.
From this it ca

Fundamentals of probability.
6.436/15.085
LECTURE 8
Continuous random variables.
8.1. Continuous random variables
Recall that we dened a r.v. X : R to be continuous if there exists an (Riemann) integrable
function f : R [0, ) such that the distribution fu

Fundamentals of probability.
6.436/15.085
David Gamarnik
LECTURE 2
Extension theorem. Uniform (Lebesgue) probability
measure and innite coin tosses model
Outline of Lecture
Extension theorem.
Application of extension theorem: uniform measure, innite coi

Advanced Stochastic Processes.
David Gamarnik
LECTURE 3
Large Deviations for i.i.d. Random Variables
Outline of Lecture
Cherno bound using exponential moment generating functions. Examples. Legendre
transforms.
3.1. Preliminary notes
The Weak Law of Larg

Fundamentals of probability.
6.436/15.085
David Gamarnik
LECTURE 1
Probability basics: probability space, -eld, probability
measure, examples.
Outline of Lecture
Probabilistic experiments
Discrete discrete probability space.
Probability space (, F, P)

Fundamentals of probability.
6.436/15.085
LECTURE 5
Random variables and distribution functions
Outline of Lecture
Random variables and distribution functions.
Discrete and continuous random variable.
5.1. Random variables
Often in probabilistic experim

Advanced Stochastic Processes.
David Gamarnik
LECTURE 11
Martingales and stopping times. Applications
Lecture outline
Second stopping theorem.
Doob-Kolmogorov inequality.
Applications of stopping theorems to hitting times of a Brownian motion.
11.1. Se

Fundamentals of probability.
6.436/15.085
LECTURE 15
Transforms and moment generating functions
15.1. Moment generating function
Moment generating function, transform and characteristic functions are convenient forms of
representing entire probability dis

Fundamentals of probability.
6.436/15.085
LECTURE 13
Modes of convergence. Laws of Large Numbers
13.1. Introduction
Recall that we dened E[X] for an arbitrary r.v. X by approximating it using simple r.v. Xr .
The sequence Xr was converging to X in the fol

Fundamentals of probability.
6.436/15.085
LECTURE 16
Branching processes.
16.1. Preliminaries
So far we have been considering the derivatives of the moment generating functions without
much regard to why the derivatives exist. In this lecture we introduce

Fundamentals of probability.
6.436/15.085
LECTURE 6
Discrete random variables and expectations
Outline of Lecture
Discrete random variables.
Conditional marginal and conditional distributions.
Expectations.
6.1. Preliminaries: random variables in Rd .

Fundamentals of probability.
6.436/15.085
LECTURE 10
Continuous random variables III. Derived distributions.
Convolution of random variables.
10.1. Functions of random variables. Change of variables technique
Given a continuous r.v. X with density f and a

Fundamentals of probability.
6.436/15.085
LECTURE 14
Modes of convergence II.
14.1. Introduction
We continue the discussion of various modes of convergence of r.v. We introduce two more
modes of convergence - in distribution and in expectation, or in gene

Fundamentals of probability.
6.436/15.085
LECTURE 23
Birth-death processes
23.1. General birth-death processes
An important and a fairly tractable class of innite continuous time M.c. is a birth-death process.
Loosely speaking this is a process which comb

Fundamentals of probability.
6.436/15.085
LECTURE 24
Walds identity
24.1. Walds equation
Theorem 24.1. Let Xn , n 1 be an i.i.d sequence with E[X1 ] = < . Let be a stopping
time with E[ ] < . Then E[ 1i Xi ] = E[ ].
Proof. Note
1i
Xi =
i1
Xi 1cfw_ i. Also

Fundamentals of probability.
6.436/15.085
LECTURE 19
Markov chains
19.1. Introduction
d
Recall a model we considered earlier: random walk. We have Xn = Be(p), i.i.d. Then Sn =
1jn Xj was dened to be a simple random walk. One of its key property is that th

Fundamentals of probability.
6.436/15.085
LECTURE 20
Markov chains II. Mean recurrence times.
20.1. Markov chains with a single recurrence class
Recall the relations , introduced in the previous lecture for the class of nite state Markov
chains. Recall th

Fundamentals of probability.
6.436/15.085
LECTURE 17
Characteristic Functions. Law of Large Numbers.
Central Limit Theorem
17.1. Introduction
The power of characteristic functions comes from the fact that they usually uniquely characterize
the underlying

Fundamentals of probability.
6.436/15.085
LECTURE 20
Markov chains III. Periodicity, Perron-Frobenius,
Absorption
20.1. Periodicity
In Lecture 20, we showed that E[Ni (t)|X0 = k]/t i as t , irrespective of the starting
state k. Since Ni (t) = t 1cfw_Xn =i