Example: Consider an RC circuit as shown. Find h(t)
for this circuit.
R
+
+
x(t)
i(t)
C

y(t)

Write KVL,
1
C
R i(t) +
t
i(t) dt = x (t)
Take FT of both the sides,
R I ( ) +
1
I ( ) = X ( )
j C
Department of Electrical & Computer Engineering
I ( ) =
A
Chapter 7: Spectral Density
Although Fourier transform does not exist for r.p.s (infinite
energy), the autocorrelation and crosscorrelation functions
are nonperiodic and energy signals. Thus for these functions,
Fourier transform does exist. This fouri
Brief Review on Digital Filters:
Why Digital vs. Analog?
Advantages:
Less (no) sensitivity to temperature variations / aging.
Better reliability.
Can be made adaptive.
More compact and light weight.
Less cost, etc.
Disadvantage:
Quantization effects due t
Time Averages and Ergodicity:
In practice, we would like to deal with only a single sample
function rather than the ensemble of functions. For example,
we may wish to infer the probability law or certain averages
of the r.p. from the measurements on a sin
Measures on two (or more) r.p.s:
Let X(t) and Y(t) be r.p.s with ACFs RXX(t 1,t2 ) and RYY(t1,t 2),
respectively. The crosscorrelation between X(t) and Y(t),
which gives a measure of dependence between these processes
is,
RXY ( t1 , t 2 ) = E[ X (t1)Y (
Chapter 5: Random Processes
In engineering and science, we often encounter problems, which
involve random signals, e.g., bit stream in a binary communication
system (random message), noise or other interferences, etc.
So far, we have investigated random e
Example 12: Given joint PDF
f X ,Y ( x, y ) = 0.4 ( x + ) ( y 2) + 0.3 ( x ) ( y 2) +
0.1 ( x ) ( y ) + 0.2 ( x 1) ( y 1)
Find , which minimizes the correlation rX,Y.
Find minimum r X,Y.
Are X, Y orthogonal?
fX,Y(x,y)
0.4

y
0.2 0.3
x
0.1
Department of E
Note that,
20 = E[( X X ) 2 ] = 2
X
2
02 = E[(Y Y ) 2 ] = Y
The other 2nd order joint central moment, which is very
important is 11. This is called as the Covariance of X and Y.
c XY = 11 = E[( X X )(Y Y )]
Expanding the term in the bracket gives,
c XY =
Operations on Multiple r.v.s:
Joint Moments, Correlation and Covariance:
These describe the relationships between r.v.s.
(1) Moments about the origin:
The expected value of product XiYj is the joint moment of
order i + j, i.e.,
[
mi , j = E X iY
j
]= x i
Example 5: Given
1
f X ,Y ( x , y ) = r 2
0
x 2 + y2 r2
elsewhere
Find f X(xy).
Example (5): Use marginal density,
fY ( y ) =
r 2 y2
dx
2
r 2 y 2 r
f X ,Y ( x, y ) dx =
2 r 2 y 2
=
r2
0
y < r
y > r
Department of Electrical & Computer Engineering
1
Definition2 (Joint Probability Density Function):
For two r.v.s X and Y, the joint PDF is
f X ,Y ( x , y ) =
2 FX ,Y ( x, y )
x y
If X and Y are discrete r.v.s, we get
NM
f X ,Y ( x , y ) = P( xm , ym ) ( x xn ) ( y yn )
n=1 m=1
i.e., a series of Dirac
Chapter 3
Multiple Random Variables:
Many experiments involve dealing with multiple r.v.s and one
may be interested in their interactions or joint behavior, e.g.,
suppose in every high school, the SAT score and the ending
GPA for each student, S, are reco
(4) Poisson Distribution:
If a r.v. X is discrete, taking values at points k = 0, 1, with
P( X = k ) = e
a
ak
,
k!
a>0
Then X has a poisson density and distribution as,
f X ( x ) = P( X = k ) ( x k )
k =0
ak
(x k )
k = 0 k!
= e a
The CDF will be,
Depart
Important PDFs
(1) Normal (Gaussian) Distribution:
A r.v. X is called Gaussian if its PDF is of a form
1
f X (x ) =
2
2 X
e ( x x )
X: Mean
X: Standard Deviation
about the mean
2 2
X
2
FX (x)
fX (x)
1
2
2 X
1
0.841
X > Y
0. 607
0.5
2
2 X
0.159
X
X
X
X +
Statistical Averages and Expectation
It is often desirable to characterize a r.v. by a few numbers
(deterministic). These are called moments of r.v. and are
obtained by certain averaging or expectation operation.
Consider, for example, the average class g
Transformation of a Random Variable
Let X be a r.v. with known PDF, fX (x), or CDF, FX (x). This
r.v., is transformed using transformation function T(.) to
r.v. Y = T(X)
X
Y = T (X)
fX (x)
T (.)
fY (y)
We would like to determine f Y (y). Three cases are c
CDF of Discrete r.v. :
A discrete r.v., X taking on one of the countable set of possible
values x1, x2 with probability P [X = xk], k [1, N]
forming a stairstep CDF with amplitude of each step being
P [X = xk], k = 1, 2 . Thus,
N
FX (x) = P[X = x k ] u (
Chapter 2
Random Variables:
Many random phenomena have outcomes that are real numbers,
e.g., the voltage, v(t) at time, t, across a noisy resistor, number
of people on a New York to Chicago train, etc. In engineering
and science, we are generally interest
Independent Events:
Events A and B are said to be statistically independent if
P( A  B) = P( A)
i.e., the prob. of event A is the same whether event B has
occurred or not. Thus the equivalent condition is
P( A  B) =
P ( A B)
= P( A)
P ( B)
P( A B) = P(
Remark:
The total probability is still valid even if the union of Bn does
not equal S, provided that
N
A U Bn
n=1
Bayes Theorem:
Recall that conditional probability applies to any two events,
P( A  B) =
P( B  A) =
P( A B)
P ( B)
P( B A) P ( A B)
=
P ( A
Remark:
The total probability is still valid even if the union of Bn does
not equal S, provided that
N
A U Bn
n=1
Bayes Theorem:
Recall that conditional probability applies to any two events,
P( A  B) =
P( B  A) =
P( A B)
P ( B)
P( B A) P ( A B)
=
P ( A
(3) If A C = , then
Pr( A C  B ) = Pr( A  B ) + Pr( C  B)
Proof: Use the equation for Conditional Probability,
Pr( A C  B ) =
Pr( A C B)
Pr( B)
Now using distributive law,
( A C ) B = ( A B) ( C B)
But, ( A B) ( C B) = A B C
= AC B
=
1
Department of E
Probability Definition and Axioms
Probability is a nonnegative number assigned to an event, A,
and is denoted by Pr(A).
Axioms of Probability (Kolmogrov):
(1) Pr (A) 0
(2) Pr (S) = 1
, i.e., Certain.
(3) If A B = , i.e., mutually exclusive events, then
P
Set Definitions
Set: Collection of objects, which are called set elements.
aA
a is element of set A
aA
a is not an element of set A
In study of probability, elements are outcomes of experiments.
= cfw_1, 2 ,
where i = i th outcome
Empty Set: Contains no
Introduction to Probability
Deterministic vs. Random Phenomena:
Deterministic: Outcome is certain
Random: Outcome is uncertain
e.g.
Background hiss noise in radio broadcast, Motion of
electrons, etc.
X(t) = A cos t
,
A, : fixed
is deterministic. But if we