of Definition a Random Variable
Random Variable [m-w.org]
: a variable that is itself a function of the result of a statistical
experiment in which each outcome has a definite
probability of occurrence
Copyright Syed Ali Khayam 2009
3
Definition of a Random Variable
A random variable is a mapping from an outcome s of a
random experiment to a real number
X : S SX
domain
range
SX is called the image of X
called the
Copyright Syed Ali Khayam 2009
4
Definition of a Random Variable
X : S SX
Random
Experiment
Sample Space
Random
Variable
X(s)
head
tail
0
1
R
Sx
Copyright Syed Ali Khayam 2009
5
Definition of a Random Variable
X : S SX
X(s)
123456
Random
Experiment
Experiment
Sample Space, S
R
Sx
Random Variable
Image courtesy of www.buzzle.com/
Copyright Syed Ali Khayam 2009
6
Definition of a Random Variable
More than one outcomes can be mapped to the same real
number
X : S SX
X(s)
0
Random
Experiment
1
Sx
Sample Space
Random Variable
Image courtesy of www.buzzle.com/
Copyright Syed Ali Khayam 2009
7
Types of Random Variables
Discrete random variables: have a countable (finite or infinite)
image
Sx = {0, 1}
Sx = {, -3, -2, -1, 0, 1, 2, 3, }
Continuous random variables: have an uncountable image
Sx = (0, 1]
Sx = R
Mixed random variables: have an image which contains
continuous and discrete parts
Sx = {0} U (0, 1]
We will mostly focus on discrete and continuous random
variables
Copyright Syed Ali Khayam 2009
8
Di
Discrete Random Variables
Copyright Syed Ali Khayam 2009
9
Probability Mass Function
The Probability Mass Function (pmf) or the discrete probability
density function provides the probability of a particular point in
the sample space of discrete random variable (rv)
the sample space of a discrete random variable (rv)
For a countable SX={a0, a2, , an}, the pmf is the set of
probabilities
pX (ak ) = Pr {X = ak }, k = 1, 2, , n
pX(ak)
SX={a0=0, a1=1, , a5=5},
0
Copyright Syed Ali Khayam 2009
1
2
3
4
5
X
10
Properties of a PMF
P1: 0 pX (ak ) 1
P2:
p
X
(ak ) = 1
ak S X
pX(ak)
SX={a0=0, a1=1, , a5=5},
0
Copyright Syed Ali Khayam 2009
1
2
3
4
5
X
11
Cumulative Distribution Function (CDF) of Discrete
Random Variable
The Cumulative Distribution Function (CDF) for a discrete rv is
defined as:
FX (t ) = Pr {X t } = pX (x )
x t
pX(x)
FX(x)
pmf
1
2
Copyright Syed Ali Khayam 2009
3
CDF
4
X
1
2
3
4
X
12
Cumulative Distribution Function (CDF) of Discrete
Random Variable
CDF can be used to find the probability of a range of values in a
rvs image:
Pr {a < X b } = Pr {X b } Pr {X a }
= FX (b ) FX (a )
pX(x)
FX(x)
pmf
1
2
Copyright Syed Ali Khayam 2009
3
CDF
4
X
1
2
3
4
X
13
Properties of a CDF
F1: 0 FX (x ) 1
F2: a b
< x <
FX (a ) FX (b )
pX(x)
FX(x)
pmf
1
2
Copyright Syed Ali Khayam 2009
3
CDF
4
X
1
2
3
4
X
14
Properties of a CDF
F3:
limx FX (x ) = 0
limx FX (x ) = 1
F4: FX (x i +1 ) = FX (x i ) + pX (x i +1 )
pX(x)
FX(x)
pmf
1
2
Copyright Syed Ali Khayam 2009
3
CDF
4
X
1
2
3
4
X
15
Expected Value of a Discrete Random
Variable
The expected value, expectation or mean of a discrete rv is the
average value of the random variable
What is the average value of a random variable whose image is
SX={1, 6, 7, 9, 13}?
Copyright Syed Ali Khayam 2009
16
Expected Value of a Discrete Random
Variable
What is the average value of the following random variable
whose image is SX={1, 6, 7, 9, 13}?
If your answer is 7.2 then you assumed that all of the values in the
rvs image have equal weights
1
1
1
1
1
1
1 + 6 + 7 + 9 + 13 = (1 + 6 + 7 + 9 + 13) = 7.2
5
5
5
5
5
5
Copyright Syed Ali Khayam 2009
17
Expected Value of a Discrete Random
Variable
Mathematically, the expected value of a discrete random
variable is:
{X } = X =
a
k
Pr {X = ak }
ak S X
In some cases, the expected value does not converge
some cases the expected value does not converge
In such cases, we say that the expected value does not exist
Copyright Syed Ali Khayam 2009
19
Variance of a Random Variable
Variance of a rv is a measure of the amount of variation of a rv
around its mean
Intuitively, which of the following discrete rvs has a higher
variance?
pX(x)
qX(x)
E{X}=3.87
0.5
0.4
E{X}=5.2
0.4
0.2
0.1
0.033
1
6
Copyright Syed Ali Khayam 2009
7
9
13
X
1
6
7
9
13
X
20
Variance of a Random Variable
Intuitively, which of the following discrete rvs has a higher
variance?
qX(x) has a higher variance because it varies more around its mean
than pX(x)
pX(x)
qX(x)
E{X}=3.87
0.5
0.4
E{X}=5.2
0.4
0.2
0.1
0.033
1
6
Copyright Syed Ali Khayam 2009
7
9
13
X
1
6
7
9
13
X
21
Variance of a Random Variable
Mathematically, the variance of a discrete rv is defined as:
var {X } = =
2
X
(a
2
k
{X }) Pr {X = ak }
ak S X
Copyright Syed Ali Khayam 2009
22
Variance of a Random Variable
pX(x)
qX(x)
E{X}=3.87
0.5
0.4
E{X}=5.2
0.4
var{X}=15.36
0.2
var{X}=9.91
0.1
0.033
1
6
7
9
13
X
2
7
9
13
X
2
2
(6 3.87) 0.4 + (7 3.87) 0.033 +
2
6
var {X } = (1 5.2) 0.4 +
var {X } = (1 3.87) 0.5 +
2
1
2
2
2
(6 5.2) 0.2 + (7 5.2) 0.2 +
2
2
(9 3.87) 0.033 + (13 3.87) 0.033
(9 5.2) 0.1 + (13 5.2) 0.1
= 9.91
= 15.36
Copyright Syed Ali Khayam 2009
23
Standard Deviation of a Random Variable
In many scenarios, we use the square root of the variance
called its standard deviation
X = var {X }
Copyright Syed Ali Khayam 2009
24
Standard Deviation of a Random Variable
pX(x)
qX(x)
E{X}=3.87
0.5
0.4
E{X}=5.2
0.4
X=3.92
0.2
X=3.14
0.1
0.033
1
6
7
9
13
X
X = var {X } = 9.91 = 3.14
Copyright Syed Ali Khayam 2009
1
6
7
9
13
X
X = var {X } = 15.36 = 3.92
25
Discrete Uniform Random Variable
A discrete uniform rv, D, has a finite image and all the elements
of the image have equal probabilities
Pr{D=k}
1/n
x1
x2
x3
xn
k
What do you think are the expected value and standard
deviation of this random variable?
Copyright Syed Ali Khayam 2009
26
Bernoulli Random Variable
A Bernoulli Random Variable is defined on a single event A
This rv is based on an experiment called a Bernoulli trial
The experiment is performed and the event A either happens or does not
happen
Thus the sample space of a Bernoulli rv is binary
the sample space of Bernoulli rv is binary
B
Bernoulli Trial: Toss a
r
a
coin
Sample Space
Bernoulli Random
Variable
X(s)
A=head
Ac = Not head
= tail
0
1
R
Sx
Copyright Syed Ali Khayam 2009
27
Bernoulli Random Variable
A Bernoulli Random Variable is defined on a single event A
This rv is based on a the experiment called a Bernoulli trial
The experiment is performed and the event A either happens or does not
happen
Thus the sample space of a Bernoulli rv is binary
Pakistan cricket team
a
c
plays Australia
u
Bernoulli Random
Variable
X(s)
A=Pak wins
R
0
Ac=Pak losses
1
Sample Space
Image Courtesies of http://www.tribuneindia.com and
images.google.com
Copyright Syed Ali Khayam 2009
28
Bernoulli Random Variable
A Bernoulli Random Variable is defined on a single event A
This rv is based on a the experiment called a Bernoulli trial
The experiment is performed and the event A either happens or does not
happen
Thus the sample space of a Bernoulli rv is binary
Of course the Pr{A} = 0 for this
Bernoulli Random
experiment
Pakistan cricket team
n
e
plays Australia
s
a
Copyright Syed Ali Khayam 2009
Variable
X(s)
A=Pak wins
R
Ac=Pak losses
0
1
Sample Space
Image Courtesies of http://www.tribuneindia.com and
images.google.com
29
Bernoulli Random Variable
Typical examples of Bernoulli rvs in communication:
Transmit a bit over a wireless channel
Outcomes:
0 > bit is received error-free
1 > bit received is not received error-free => bit is received with errors
Transmit a packet over the Internet
packet over the Internet
Outcomes:
0 > packet is received
1 > packet is not received => packet is lost en-route due to congestion
Copyright Syed Ali Khayam 2009
30
Bernoulli Random Variable
Sample space of a Bernoulli rv, I, is binary
Both outcomes are mapped to real numbers,
Traditionally: I(A) = 1 and I(Ac) = 0 are used to represent a Bernoulli rvs
outcomes
The pmf of I is:
Pr{I = 1} = p
Pr{I = 0} = 1 p
Pr{I=k}
p
1-p
0
Copyright Syed Ali Khayam 2009
1
I
31
Bernoulli Random Variable
Example:
Consider the experiment of a fair coin toss. What is the expected
value and the variance of this Bernoulli rv?
Pr{I=k}
0.5
0
Copyright Syed Ali Khayam 2009
1
I
32
Discrete Random Variables
Example:
Consider the experiment of a fair coin toss. What is the expected
value and the variance of this Bernoulli rv?
th
thi
Since the coin toss is fair: Pr{I = 1} = 0.5 and Pr{I = 0} = 0.5
E{I} = (1)x(0.5) + (0)x(0.5) = 0.5
(1)x(0
(0)x(0
var{I} = (10.5)2x(0.5) + (00.5)2x(0.5) = 0.25
I=0.5
Pr{I=k}
I=0.5
0.5
0
Copyright Syed Ali Khayam 2009
1
I
33
Bernoulli Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error
0.1. What is the expected value and the variance of this Bernoulli
rv?
Pr{I=k}
0.9
0.1
0
Copyright Syed Ali Khayam 2009
1
I
34
Bernoulli Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error 0.1. What is the
expected value and the variance of this Bernoulli rv?
The event of interest here is a bit-error:
=> Pr{I = 1} = 0.9 and Pr{I = 0} = 0.1
E{I} = (1)x(0.9) + (0)x(0.1) = 0.9
(1)x(0
(0)x(0
var{I} = (10.9)2x(0.9) + (00.9)2x(0.1) = 0.09
Note that the variance of this pmf is smaller than the variance of the coin toss
pmf
Pr{I=k}
0.9
I=0.3
I=0.9
0.1
0
Copyright Syed Ali Khayam 2009
1
I
35
Binomial Random Variable
Consider a collection of n independent Bernoulli trials
A Binomial Random Variable is the total number of occurrences
of an event A in this independent Bernoulli collection
Send
Send n bits, count the number of bits that are received with errors
count the number of bits that are received with errors
Send n packets, count the number of packets that are not lost
Copyright Syed Ali Khayam 2009
36
Binomial Random Variable
If Ij(A)=1 and Ij(Ac)=0, j=1,2,,n, are used to represent the
outcomes of the Bernoulli trials then the Binomial Random
Variable, X, is
n
X = Ij
j =1
So what is the image of X?
Copyright Syed Ali Khayam 2009
37
Binomial Random Variable
If Ij(A)=1 and Ij(Ac)=0, j=1,2,,n, are used to represent the
outcomes of the Bernoulli trials then the Binomial Random
Variable is
Variable, X, is
n
X = Ij
j =1
So what is the image of X?
SX = {0, 1, 2, 3, , n}
Copyright Syed Ali Khayam 2009
38
Binomial Random Variable
Pr{Ij(A)=1} = p and Pr{Ij(A)=0} = 1-p
Then a Binomial rv X is defined as:
n
X = Ij
j =1
And the pmf of a binomial rv is
bi
n k
p (1 p )n k
Pr {X = k } =
k
Copyright Syed Ali Khayam 2009
39
Binomial Random Variable
The pmf of a Binomial rv X is:
n k
p (1 p )n k
b (k ; n, p ) = Pr {X = k } =
k
This pmf gives the probability that exactly k out of a total of n
pmf gives the probability that exactly
of total of
Bernoulli trials were successes
Be very careful about the definition of a success
Copyright Syed Ali Khayam 2009
40
Binomial Random Variable
Pr{X=k}
k
Image courtesy of Wikipedia article on Binomial Distribution
Copyright Syed Ali Khayam 2009
41
Binomial Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error p.
Fi th
Find the probability that a packet of n bits is received with one
th
or more errors?
Copyright Syed Ali Khayam 2009
42
Binomial Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error p.
Fi th
Find the probability that a packet of n bits is received with one
th
or more errors?
Pr{Ij = 1} = 0.1 and Pr{Ij = 0} = 0.9, j=1,2,,n
Then the probability that a packet is received with errors is
Pr {(X = 1) (X = 2) (X = 3) (X = n )}
Copyright Syed Ali Khayam 2009
43
Binomial Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error p.
Fi th
Find the probability that a packet of n bits is received with one
th
or more errors?
Pr {pkt with errs} = Pr {(X = 1) (X = 2) (X = 3) (X = n )}
= Pr {X = 1} + Pr {X = 2} + Pr {X = 3} + + Pr {X = n }
n
n
n
n 1
p (1 p )n 1 + p 2 (1 p )n 2 + p 3 (1 p )n 3 + + p n (1 p )n n
=
n
1
2
3
n n
n i
= p i (1 p )
i
i =1
Copyright Syed Ali Khayam 2009
44
Binomial Random Variable
Example:
Consider a binary symmetric channel with probability of bit-error p.
Fi th
Find the probability that a packet of n bits is received with one
th
or more errors?
n i
n i
Pr {pkt with errs} = p (1 p )
i
i =1
n
There is an easier way to compute the same probability by noting
that:
Pr {pkt with errs} = Pr {X > 0} = 1 Pr {X < 0} = 1 Pr {X = 0}
n 0
n 0
= 1 p (1 p )
0
n
= 1 (1 p )
Copyright Syed Ali Khayam 2009
45
Connection Between Bernoulli and Binomial
RVs
I(A)=1, I(A)=0
S
n Bernoulli trials
SI
A
Pr{A}
SX
Pr{I=1}=p
Pr{I=0}=1-p
Bernoulli Trial
Copyright Syed Ali Khayam 2009
Bernoulli RV
n k
n k
Pr {X = k } = p (1 p )
k
Binomial RV
46
Geometric Random Variable
A Geometric Random Variable, M, is the number of Bernoulli
trials until the first occurrence of an event A
The experiment is stopped as soon as event A is observed
The image of a Geometric random variable is infinite but
countable
SM = {1, 2, 3, }
{1
Copyright Syed Ali Khayam 2009
48
Geometric Random Variable
k 1
pZ (k ) = Pr {Z = } k = (1 p )
p
OR
k
pZ (k ) = Pr {Z = k } = (1 p ) p
Depending on whether the success trial is included in the total
count or not
Copyright Syed Ali Khayam 2009
Also called the modified
l
o
geometric pmf
o
m
The pmf of a Geometric Random Variable, Z, is
49
Geometric Random Variable
Image courtesy of Wikipedia article on Geometric Distribution
Copyright Syed Ali Khayam 2009
50
Connection between Geometric and Binomial
RVs
Can we find the probability of a Geometric random variable
using the Binomial random variable?
k
Pr {Z = k } = (1 p ) p
n k
p (1 p )n k
Pr {Z = k } =
k
Copyright Syed Ali Khayam 2009
Geometric
Binomial
51
Connection between Geometric and Binomial
RVs
Can we find the probability of a Geometric random variable using
the Binomial random variable?
k
p1 1 p k 1
Pr {X = k } = (
)
1
k 1
Pr {X = k } = (1 p )
Copyright Syed Ali Khayam 2009
k
1
p = Pr {Z = k }
52
Connection between Geometric and Binomial
RVs
In k Bernoulli trials, there are k ways in which you can have 1
success and k-1 failures
Since the Binomial random variable counts successes and
failures, it sums and considers all the k outcomes together
Copyright Syed Ali Khayam 2009
53
Connection between Geometric and Binomial
RVs
For k=4, a Binomial random variable X jointly considers the
outcomes 0001, 0010, 0100, 1000
Pr{X = 1} = Pr{(0001) U (0010) U (0100) U (1000)}
1} Pr{(0001) (0010) (0100) (1000)}
Pr{X = 1} = Pr{0001} + Pr{0010} + Pr{0100} + Pr{1000}
Since the underlying Bernoulli trials are independent:
Pr{X = 1} = (k)Pr{one success in k trials}
For the Geometric random variable, we are only interested in
the Geometric random variable we are only interested in
one of these outcomes, 0001
=> Pr{Z = 1} = Pr{X = 1} / k
Copyright Syed Ali Khayam 2009
54
Memoryless Property
It can be shown that the Geometric rv satisfies the memoryless
property
The memoryless property is satisfied when:
Pr {Z = j + k Z k } = Pr {Z = j }
This property is also called the Markov Property
Copyright Syed Ali Khayam 2009
55
Memoryless Property
The memoryless property is satisfied when:
Pr {Z = j + k Z k } = Pr {Z = j }
For the Geometric rv, RHS of the above equation is:
j
Pr {Z = j } = (1 p ) p
Thus to prove that the Geometric rv satisfies the memoryless
to prove that the Geometric rv satisfies the memoryless
property, we need to show that
Pr {Z = j + k Z k } = (1 p ) p
j
Copyright Syed Ali Khayam 2009
56
Memoryless Property
To prove that the Geometric rv satisfies the memoryless
property, we need to show that
Pr {Z = j + k Z k } = (1 p ) p
j
Lets expand the LHS using the definition of conditional
probability
Pr {Z = j + k Z k } =
Pr {Z = j + k Z k }
Pr {Z k }
Pr {Z = j + k }
=
Pr {Z k }
j +k
(1 p ) p
=
k
k +1
k +2
(1 p ) p + (1 p ) p + (1 p ) p +
Copyright Syed Ali Khayam 2009
57
Memoryless Property
Continued from last page
j +k
(1 p ) p
Pr {Z = j + k Z j } =
k
k +1
k +2
(1 p ) p + (1 p ) p + (1 p ) p +
j +k
=
(1 p )
p
(1 p ) ((1 p ) p + (1 p ) p + (1 p ) p + )
k
0
1
2
j +k
(1 p ) p
=
k
(1 p )
j
= (1 p ) p
Summation over all possible values
of the Geometric pmf
Copyright Syed Ali Khayam 2009
58
Memoryless Property
It can also be shown that the Geometric rv is the only discrete
random variable that satisfies the memoryless property
Because of the memoryless property, the Geometric rv can be
thought of as the number of failures between two successes
OR the inter-arrival time between successes
Copyright Syed Ali Khayam 2009
59
Poisson Random Variable
A Poisson Random Variable, N, is the number of occurrences or
arrivals of an event A in a time interval of fixed length
E.g.: The number of packets that arrive at a wireless access point per
The number of packets that arrive at wireless access point per
second
The Poisson rv assumes that:
Poisson rv
that:
The average number of arrivals per time interval, denoted by , is known
The arrivals are independent of each other
Poisson time interval
t
Poisson arrivals
Copyright Syed Ali Khayam 2009
60
Poisson Random Variable
A good way of understanding Poisson rv is through the Binomial
rv
Let us divide the fixed Poisson time interval into n very small
sub-intervals of length t
t is so small that only one arrival is possible within each sub-interval
Poisson time interval
1234
t
Copyright Syed Ali Khayam 2009
n
t
Poisson arrivals
61
Poisson Random Variable
Now the experiment can be treated as Binomial rv where a
success corresponds to the presence of an arrival in a subinterval
Given that is the average arrival rate, what is the probability of
success (success corresponds to an arrival)?
Poisson time interval
1234
t
Copyright Syed Ali Khayam 2009
n
t
Poisson arrivals
62
Poisson Random Variable
Now the experiment can be treated as Binomial rv where a
success corresponds to the presence of an arrival in a subinterval
Given that is the average arrival rate, what is the probability of
success (success corresponds to an arrival)?
Pr{success} = p = /n
Poisson time interval
1234
t
Copyright Syed Ali Khayam 2009
n
t
Poisson arrivals
63
Poisson Random Variable
Now the experiment can be treated as Binomial rv with
parameter /n
k
n k
n
Pr {X = k } = b k ; n, = 1
k
n n
n
k
n!
=
(n k ) ! k ! n
n k
1
n
n n 1 n 2
=
n
n n
n
k
n k + 1 k
1 1
k !
n
n
n
Poisson time interval
time interval
1234
t
Copyright Syed Ali Khayam 2009
n
t
Poisson arrivals
64
Poisson Random Variable
Thus for large n, the Binomial pmf approaches the Poisson pmf
k
lim Pr {X = k } = lim b k ; n, =
k!e
n
n
n
This is called the Poisson approximation of Binomial random
is called the Poisson approximation of Binomial random
variable
Poisson time interval
Poisson time interval
1234
t
Copyright Syed Ali Khayam 2009
n
t
Poisson arrivals
66
Poisson Random Variable
In general, the pmf of a Poisson random variable is
k exp { }
Pr {N = k } =
k!
where is the average arrival rate (arrivals/time interval)
Copyright Syed Ali Khayam 2009
67
End of Discrete Random Variables
At this point, we conclude our discussion on examples of
discrete random variables
We will cover some other aspects of discrete rvs after looking at
examples of continuous rvs
examples of continuous rvs
Copyright Syed Ali Khayam 2009
69
Continuous Random Variables
Copyright Syed Ali Khayam 2009
72
Continuous Random Variables
Recall that a Continuous random variable has an uncountable
image
Sx = (0, 1]
Sx = R
Copyright Syed Ali Khayam 2009
73
Probability Density Function (pdf) of a
Continuous Random Variable
The Probability Density Function (pdf) of a continuous random
variable, fX(x), is a continuous function of the x Sx
Properties of pdf:
f1: fX (x ) 0, x
f2:
fX (x )dx = 1
fX(x)
SX=(0, n]
0
Copyright Syed Ali Khayam 2009
n
x
74
Probability Density Function (pdf) of a
Continuous Random Variable
f3:
Pr {a X b } = Pr {a X < b } = Pr {a < X b } = Pr {a < X < b }
b
=
f
X
(x )dx
a
fX(x)
0
Copyright Syed Ali Khayam 2009
a
b
n
x
75
Cumulative Distribution Function (CDF) of a
Continuous Random Variable
CDF of a continuous rv is computed by integrating the pdf
t
FX (t ) = Pr {X t } =
fX (x )dx
x =
Conversely, we also have:
fX
(x )
d
FX (t )
dx
FX(x)
x
Copyright Syed Ali Khayam 2009
76
Properties of the CDF of a Continuous
Random Variable
F1:
0 FX (x ) 1,
F2:
a b
F3:
< x <
FX (a ) FX (b )
limx FX (x ) = 0
limx FX (x ) = 1
FX(x)
x
Copyright Syed Ali Khayam 2009
77
Properties of the CDF of a Continuous
Random Variable
a
F4: Pr {X = a } = Pr {a X a } =
f
X
(x ) dx
=0
a
FX(x)
x
Copyright Syed Ali Khayam 2009
78
Properties of the CDF of a Continuous
Random Variable
F4:
{
}
a + 2
Pr a X a +
= fX (x ) dx fX (a )
2
2
a 2
for small values of .
FX(x)
a-/2 a
Copyright Syed Ali Khayam 2009
a+/2
x
79
Properties of the CDF of a Continuous
Random Variable
F5:
Pr {a X b } = Pr {a X < b } = Pr {a < X b } = Pr {a < X < b }
= FX (b ) FX (a )
FX(x)
FX(b)
Pr{a<x<b}
FX(a)
a
Copyright Syed Ali Khayam 2009
b
x
80
Expected Value of a Continuous Random
Variable
The expected value of a continuous random variable is:
{X } = X =
xf
X
(x ) dx
fX(x)
fX(x)
E{X}
x
Copyright Syed Ali Khayam 2009
E{X}
x
81
Variance of a Continuous Random Variable
The variance of a continuous random variable is:
(x
var {X } = =
2
X
X
2
) fX (x )dx
X > Y
fX(x)
X
fY(y)
x
Copyright Syed Ali Khayam 2009
Y
y
82
Exponential Random Variable
An exponential random variable measures the inter-arrival time
between two occurrences of an event
E.g., the inter-arrival time of packets arriving in a routers queue
the inter
time of packets arriving in router queue
Recall that a Poisson distribution counts the total number of
arrivals
In that sense, the exponential rv is the inter-arrival time
that sense the exponential rv is the inter
time
between two Poisson arrivals
1
2 3
4
5
6
t
Poisson arrivals
Copyright Syed Ali Khayam 2009
83
Exponential Random Variable
inter-arrival time
1234
inter-arrival time
1234
inter-arrival time
time
1234
Copyright Syed Ali Khayam 2009
=60
60
t
=20
60
t
=6
60
t
84
Exponential Random Variable
Exponential rv is the inter-arrival time between two Poisson
arrivals
Remembering the definition of a Poisson rv, what should be the
average (expected) inter-arrival time for an exponential rv, E?
1
2 3
4
5
6
t
Poisson arrivals
Copyright Syed Ali Khayam 2009
85
Exponential Random Variable
Exponential rv is the inter-arrival time between two Poisson
arrivals
Remembering the definition of a Poisson rv, what should be the
average (expected) inter-arrival time for an exponential rv, E?
E{E} = 1/
1
2 3
4
5
6
t
Poisson arrivals
Copyright Syed Ali Khayam 2009
86
Exponential Random Variable
An exponential rv is the inter-arrival time between two Poisson
arrivals
Lets look at a sub-interval (0,t] of the Poisson window
On average, how many arrivals will take place in the (0,t]
subinterval?
subinterval?
t
1
t
2 3
t
t
4
t
5
t
6
t
T
Copyright Syed Ali Khayam 2009
87
Exponential Random Variable
On average, how many arrivals will take place in the (0,t]
subinterval?
Arrivals in the (0,t] sub-interval = t/T
in the (0 sub
In other words, t arrivals will take place in a (0,t] subinterval per
window T
t
1
t
2 3
t
t
4
t
5
t
6
t
T
Copyright Syed Ali Khayam 2009
88
Exponential Random Variable
t arrivals will take place in a (0,t] subinterval per window T
(
Pr {E > t } = Pr {N = 0} =
)
0
t T e (t /T )
0!
= e t /T
FE (t ) = 1 e t /T
fE (t ) =
d
FE (t ) = e t /T
dt
T
t
1
t
2 3
t
t
4
t
5
t
6
t
T
Copyright Syed Ali Khayam 2009
89
Exponential Random Variable
Considering a window of T=1 unit time, the probability density
and cumulative distribution functions of an exponential rv are:
d
FE (t ) = e t
dt
FE (t ) = 1 e t
fE (t ) =
Like the Poisson rv, the Exponential rv is characterized by a
single parameter
single parameter
1
2 3
4
5
6
t
Copyright Syed Ali Khayam 2009
Poisson arrivals
90
Exponential Random Variable
Exponential rv is a limiting case of a geometric rv
Like the geometric rv, the exponential rv also possess the
memoryless Markov property (see textbook, Pg. 113)
Pr {E > j + k E k } = Pr {E > j }
It can be shown that exponential is the only continuous
distribution with the memoryless property
Copyright Syed Ali Khayam 2009
92
Exponential Random Variable
LHS = Pr {E > j } = 1 Pr {E j }
Memoryless property
= 1 (1 e j )
= e j
RHS = Pr {E > j + k E k } =
Pr {E > j + k E k }
Pr {E k }
Pr {E > j + k } 1 FE (E j + k )
=
=
Pr {E k }
1 FE (E < k )
=
1 (1 e ( j +k ) )
1 (1 e k )
e je k
=
e k
= e j
Copyright Syed Ali Khayam 2009
93
Uniform Distribution
A uniform rv has a uniform distribution over an interval [a, b]
1
b a , a x b
fU (x ) =
0
otherwise
0
x <a
x a
FU (x ) =
a x b
b a
1
x >b
This distribution is also called the Rectangular distribution
Copyright Syed Ali Khayam 2009
99
Uniform Distribution
fU (x)
1/(b-a)
a
Copyright Syed Ali Khayam 2009
b
x
100
Gaussian or Normal Distribution
Gaussian or Normal Distribution is by far the most famous
continuous distribution
It has the famous bell-shaped curve which can be used to
approximate many other distributions
Copyright Syed Ali Khayam 2009
101
Gaussian Distribution
The Gaussian rv is defined as:
fN
(x )
1
=
e
2
1 x 2
2
The Gaussian distribution is completely characterized by its mean
and variance and is generally written as
2
N (, )
A Gaussian distribution with zero mean and unit variance is
Gaussian distribution with zero mean and unit variance is
called a standard normal distribution
Copyright Syed Ali Khayam 2009
102
Gaussian Distribution
fN (x)
x
Image Courtesy of Wikipedia
Copyright Syed Ali Khayam 2009
103
Gaussian Distribution
The CDF of a Gaussian rv does not have a closed form
1
FN (x ) =
2
e
1 x 2
2
dt
Tables of CDF values are provided for each(, ) pair
Copyright Syed Ali Khayam 2009
104
Gaussian Distribution
Gaussian distribution does not deviate from its mean by more
than 3 standard deviations 99.7% of the times
Image Courtesy of Wikipedia
Copyright Syed Ali Khayam 2009
105