EECS 501
PROBABILITY MAPPING
Fall 2001
DEF: =sample space=set of all distinguishable outcomes of an experiment.
DEF: A=event space=set of subsets of such that A is a algebra.
DEF: An Algebra = A=a set of subsets of a set such that:
1. A A and B A A B A an

EECS 501
EXAMPLES OF SAMPLE FUNCTIONS
Fall 2001
EX #1: To generate sample functions x(n) of a Bernoulli process with p=0.6:
1. Spin a wheel of fortune once, resulting in a number o [0, 1).
2. Let o have decimal expansion 0.w1 w2 w3 w4 w5 . . . where wi =

EECS 501
RECITATION Nov. 19-21
Fall 2001
Given: Random process x(t) = A cos(t + ) where is a known constant.
Random: A and are independent random variables. f () = 21 , 0 < < 2 .
Goal: Compute the mean E [x(t)] and covariance Kx (t, s) functions of x(t).

EECS 501
ESTIMATION: MLE, MAP, LS
Fall 2001
Model: A known model of system or process with unknown parameter a.
Data: An observation R of a random variable r whose pdf depends on a.
Model fr|a (R|A): If knew a = A, would know pdf of observation r.
Goal: E

EECS 501
ESTIMATOR PROPERTIES
Fall 2001
Problem: Let cfw_x1 . . . xN be iidrv with xi N (m, 2 ) and m, 2 unknown.
Want: To compute mM LE and M LE based on observations cfw_X1 . . . XN .
2
2
2
1
N
Solution: fx1 .xN (X1 . . . XN ) = i=1 21 2 e 2 (Xi m) / s

EECS 501
Given:
With:
Note:
WSS:
DEF:
Sx (ω)
Rx (τ )
Assume:
POWER SPECTRAL DENSITY
Fall 2000
x(t) is a real-valued 0-mean WSS random process (RP).
Autocorrelation Rx (τ ) = E[x(t)x(t ± τ )] for any time t,
x(t) 0-mean→ Rx (τ ) = Kx (τ )=covariance func.

EECS 501 RESULTS SUMMARY: ERGODICITY Fall 2001
ISSUE: Let cfw_xi , i = 1, 2 . . . be a sequence of id rvs.
n
1
Does the sample mean Mn = n i=1 xi converge to
the ensemble mean E [xi ] = , and in what sense?
2
id=identically distributed; assume E [xi ], xi

EECS 501
EXAM #1
Fall 2001
PRINT YOUR NAME HERE:
HONOR CODE PLEDGE: I have neither given nor received aid on this exam, nor have I
concealed any violations of the honor code. Open book; SHOW ALL OF YOUR WORK!
SIGN YOUR NAME HERE:
(40) 1. Random variables

EECS 501
EXAM #2
Fall 2001
PRINT YOUR NAME HERE:
HONOR CODE PLEDGE: I have neither given nor received aid on this exam, nor have I
concealed any violations of the honor code. Open book; SHOW ALL OF YOUR WORK!
SIGN YOUR NAME HERE:
(30) 1. A fair (Pr[heads]

EECS 501
EXAM #3
Fall 2001
PRINT YOUR NAME HERE:
HONOR CODE PLEDGE: I have neither given nor received aid on this exam, nor have I
concealed any violations of the honor code. Open book; SHOW ALL OF YOUR WORK!
SIGN YOUR NAME HERE:
(30) 1. At 6:00 PM, EECS

EECS 501
1a. 1 = c
SOLUTIONS TO EXAM #1
1
0
dX X
X
0
dY Y = c
1
0
Fall 2001
4
2
dX X X = c X |1 c = 8.
2
80
1b. x and y are NOT independent, since pdf has nonsquare support.
HARD WAY: Compute marginal pdfs fx (X ) = fx,y (X, Y )dY and
fy (Y ) = fx,y (X, Y

EECS 501
CONTINUOUS-TIME RANDOM PROCESSES
Fall 2001
DEF: A continuous-time random process x(t) is a mapping x : RR , or:
x(t, ) : (R ) R where =sample space and R = cfw_reals.
1. Fix to R x(to , )=random variable indexed by time index to .
2. Fix o x(t, o

EECS 501
BERNOULLI AND POISSON PROCESSES
Fall 2001
DEF: Bernoulli random process x(n) is a discrete-time 1-sided iidrp with:
1 success or arrival with prob. p
p
for X = 1
x(n) =
px(n) (X ) =
0 failure or nonarrival with 1 p
1 p for X = 0
N
Note: Kolmogoro

EECS 501
COUNTABLE VS. UNCOUNTABLE SETS
Fall 2001
DEF: A set is nite if it has a nite number of elements.
DEF: Two sets A, B are in one-to-one correspondence (1-1) if
there exists a 1-1 mapping between elements of A and elements of B .
NOTE: Two nite sets

EECS 501
CONDITIONAL PROBABILITY
Fall 2001
Three There are 3 cards: red/red; red/black; black/black.
Card The cards are shued and one chosen at random.
Monte The top of the card is red. Pr[bottom is red]=?
3 possible lines of reasoning to solve this probl

EECS 501
RANDOM VARIABLES
Fall 2001
So far: Used sample space and event space A to describe outcome.
Now: Use a number to describe the outcome of an experiment.
DEF: A random variable x is a mapping x : R (=sample space).
x associates a number with each o

EECS 501
PDFs AND PMFs AND MIXTURES
Fall 2001
DEF: A discrete random variable (rv) is a mapping x : countable set.
DEF: A continuous random variable is a mapping x : uncountable set.
WLOG: Countable range of discrete rv is 1-1 with Z =integers.
DEF: The p

EECS 501
DERIVED DISTRIBUTIONS
Fall 2001
Given: pdf fx (X ) and function y = g (x). Goal: Compute pdf fy (Y ).
pmfs: py (Y ) = P r[y = Y ] = P r[g (x) = Y ] = P r[x g 1 (Y )] =
p (X ).
g 1 ( Y ) x
1
Scale: y = ax fy (Y ) = |a| fx ( Y ) so integrates to on

EECS 501
2-D EXAMPLE OF JACOBIAN METHOD
Fall 2001
Given: fx,y (X, Y ) = 6e(3X +2Y ) for X, Y 0; 0 otherwise (2-D exponential).
Goal: Compute fz,w (Z, W ) for transformation z = x + y and w = x/(x + y ).
1. Compute the inverse transformation of the given o

EECS 501
EXPECTATION
Fall 2001
DEF: The expectation = expected value = mean = 1st moment of rv x is
X px (X ) (discrete rv).
E [x] = x = X fx (X )dX (continuous rv);
Note: fx (X )=mass density E [x]=center of mass. 2 rule for linear fx (X ).
3
1. E [] is

EECS 501
COVARIANCE MATRICES
Fall 2001
DEF: A random vector is a vector of random variables x = [x1 . . . xN ] .
Note: Unless otherwise stated, a random vector is a column vector.
DEF: The mean vector of random vector x is = E [x] = [E [x1 ] . . . E [xN ]

EECS 501
CENTRAL LIMIT THEOREM
Fall 2001
DEF: rvs cfw_x1 , x2 . . . are iidrv with means and variances 2 if:
1. The xi are independent: fx1 ,x2 . (X1 , X2 . . .) = fx1 (X1 )fx2 (X2 ) . . .
2
2. xi are identically distributed: fxi (X ) = fx (X ), E [xi ] =

EECS 501
DISCRETE-TIME RANDOM PROCESSES
Fall 2001
DEF: A discrete-time random process=random sequence x(n) is mapping
x(n, ) : (Z ) R where =sample space and Z = cfw_integers.
1. Fix no Z x(no , )=random variable indexed by no .
2. Fix o x(n, o )=sample f

EECS 501
CONVERGENCE OF SEQUENCES OF RVs
Fall 2001
n > N.
Recall: nlim xn = x For any > 0, N such that |xn x| <
Given: A sequence of random variables cfw_x1 , x2 . . .. Need not be iidrv.
DEF: xn x in probability
lim
n P r [|xn
x| > ] = 0
stochastic
c