Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Random Variable
Random variable X is a mapping that maps each outcome s in the sample space to a unique
real number x, < x < .
X (s)
x
s : outcome
Sample Space
Real Line
Example
Toss a coin. Define the rv X as follows:
1 if heads
X =
0 if tails
Roll a d
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Transform Techniques  CF
[Review] Moment Generating Function
For a real t , the MGF of the random variable X is
M X (t ) E[etX ] = etX
et xk p ( x )
X k
k
=
et x f X ( x ) dx
discrete
continuous
Characteristic Function (CF)
For a real , the characte
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Counting Methods
We often find the probability of an event
by counting the number of elements in a simple sample space.
Basic methods of counting are:
Permutations
Combinations
Permutation
An arrangement of n distinct objects in a definite order is ca
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Functions of Two Random Variables
 Method by Jacobian
Suppose random variables X and Y have their joint pdf f X ,Y ( x, y ) .
Define another pair of random variables V and W
V = g1( X ,Y )
W = g2 ( X , Y )
as functions of X and Y:
.
Assume the inverse
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Sequence of Random Variables
Sample Mean
Let X be an arbitrary random variable with mean .
We want to estimate by the sample mean
X + X2 + + Xn
X = 1
n
where X1, X 2 , , X n are independent identically distributed (iid) samples of X .
is referred to as
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
EE528 Probability and Random Process
Textbook
Alberto LeonGarcia, Probability and Random Processes for Electrical Engineering, 2nd
Edition, AddisonWesley, ISBN 020150037X
Supplementary
William Feller, An Introduction to Probability Theory and Its
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Moments  part 2
Moments of Exponential RV
Let X denote the interarrival time in a Poisson arrival process with arrival rate .
Then X is an exponential random variable with pdf
f X ( x) = e x
x 0 and >0 .
The moments are
X=
1
VAR ( X ) =
1
2
CX = 1
is
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
EE210(B) Probability and Introductory Random Processes
Homework 5 Solution
1. (Problem 6.1.5)
For w = 0, 1, , 10, we observe that
P [W > w] = P [min(X, Y ) > w]
= P [X > w, Y > w]
= 0.01(10 w)2
To find the PMF of W , we observe that for w = 1, , 10,
PW (w
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
EE210(B) Probability and Introductory Random Processes
Homework 6 Solution
1. (Problem 7.1.8)
(a) The event Bi that Y = /2 + i occurs if and only if i X < (i + 1) Since
X has the uniform PDF
1/r r/2 x r/2,
fX (x) =
0
otherwise
Z (i+1)
1
P [Bi ] =
dx =
r
r
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
EE210(B) Probability and Introductory Random Processes
Homework 4 Solution
1. (Problem 5.2.5) As the problem statement says, reasonable arguments can be made
for the labels being X and Y or x and y. As we see in the argument below, the lower
case choice o
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Poisson Arrival Process
Arrivals occur
i) in a memoryless manner
ii) P [ one arrival during t ] = t + o ( t )
P [ no arrival during t ] = 1 t + o ( t )
P [ j arrivals during t ] = o ( t )
for j = 2,3,
o ( t )
=0
t 0 t
where lim
is referred to as the ar
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Functions of Two Random Variables
Maximum
Define W = max ( X ,Y ) .
Find the probability distributions of W .
Solution:
For any pair of random variables X and Y ,
FW ( w) = P [W w] = P [ X w and Y w]
= FX ,Y ( w, w)
When X and Y are independent,
FW ( w)
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Counting Methods Variations
Binomial
r
r j r j
x y
j
j =0
( x + y )r =
For the special case of x = y = 1,
r
2 =
r
r
j
j =0
For the special case of x = p and y = 1 p, for any r and p,
1=
r
r
j p j (1 p )
j =0
r j
Example
Toss a biased coin r times.
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Union of Events
Consider events A1, A2 , , AN .
The union of the events is A = A1 A2 AN .
We want to find P [ A] .
For N = 2,
P [ A] = P [ A1 ] + P [ A2 ] P [ A1 A2 ]
For N = 3,
P [ A] = P [ A1 ] + P [ A2 ] + P [ A3 ]
P [ A1 A2 ] P [ A1 A3 ] P [ A2 A3
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Funcctions of
o a Ran
ndom V
Variablee
Problem Statemeent
We knoow the pdf ( or cdf ) off a random vvariable X .
Define a new randdom variablee Y = g ( X ).
Find thhe pdf of Y .
Method
d:
Step 1: Plot Y = g ( X ).
ping the eveent Y y to a proper ev
vent
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Confidence Interval
700 samples
Sample Mean 0.3
Confidence Level
0.95
Margin of Error
0.037
We want to estimate the true mean of a random variable X economically and with confidence.
True Mean from 1M Entire Population
P X e < < X + e = Conf
Sanple Mean
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Transform Techniques
th
m
E[ X ] = X
m moment :
m
=
x m f X ( x) dx
th
m
m
m central moment : E[( X X ) ] = ( X X ) =
(x X )
m
f X ( x) dx
A convenient way of finding the moments of a random variable is the moment generating
function (MGF). Other trans
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Twoodimeension
nal Raandom
m Vecto
ors
Jointt Cumu
ulative Distrib
D
bution Functio
F
n
FX ,Y ( x, y ) P [ X x and Y y ]
Properties:
1) FX ,Y (, ) = 1
2) FX ,Y ( , y ) = FX ,Y ( x,
) = 0
3) FX ,Y ( x, y ) is a nondecreaasing functioon
4) FX ,Y ( x, )
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Some Notes on the Gaussian pdf
On the tail of the Gaussian distribution
a
0
1
2
3
4
5
6
7
8
[ref. Wozencraft and Jacob]
cdf(a)
0.5
0.841344746
0.977249868
0.998650102
0.999968329
0.999999713
0.999999999
1
1
Q(a)
0.5
0.158655254
0.022750132
0.001349898
3
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Moments of a Random Variable
The Mean of a Random Variable
E[X ] = X =
xk pk
discrete
k
x f X (x)dx
continuous
Terminology
mean = expectation =mean value = expected value
Example
Find the mean of a geometric random variable.
X is the number of times w
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Coin Tossing Games
1) Game 1 of tossing one coin.
Keep tossing a coin until A or B wins. A wins on contiguous HH, and B wins on HT.
Q: P[ A wins ] = P[ B wins ]? .
2) Game 2 of tossing one coin.
Keep tossing a coin until A or B wins. A wins on contiguou
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Introduction to the Probability Theory
The Basic Terminology
E
Experiment
The experiment describes the problem to solve.
s
Outcome
The outcome varies each time we repeat the experiment.
S
Sample Space
The set of all possible outcomes. s S always.
A
Even
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
1
Stirlings Formula
n ! ~ 2 n
n+
1
2 e n
indicates the ratio of the two sides tends to unity as n .
Primer
Maclaurin Series
log (1 + x ) = x
x 2 x3 x 4
+
+
2
3
4
1 < x < 1
( p1)
replacing x by x
1
x 2 x3 x 4
= x+
+
+
+
1 x
2
3
4
adding eq.p1 and eq.p 2,
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
MOMENTS
Mean of a Random Variable
xk pk k E[ X= X ] = x f X (x)dx  discrete
mean:
continuous
Example
Find the mean of a geometric random variable. pmf: pk =] = 1 p , k = P[ X = p ) k  k (1  1, 2,3,. 1 X = k pk = k (1  p )k 1 p = p k k =1
Example
Fin
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
Random Variable
X (s) = x
s
REAL LINE
x
Random Variable X is a mapping that maps each outcome s in the sample space to a unique real number x,  < x < . You must specify how the mapping X operates. " Let X be the random variable indicating the number of
Korea Advanced Institute of Science and Technology
probability and random process
EE EE528

Spring 2010
Cumulative Distribution Function of a Random Variable
Definition:
FX ( x) = P[ X x] = P[cfw_s :  X ( s ) x]
Other terms are cdf and Probability Distribution Function (PDF).
Example Uniform Distribution
Throw dart at a spinning wheel. X is the phase where