Notes on Probability and Random Variables
Optimal Estimation (EML 6934, Section 6385)
Fall 2009
University of Florida, Mechanical and Aerospace Engineering
Prabir Barooah
*
1
Probability
1.1
Random Experiment
Everything is based on a “random experiment”, which is a mathematical model for some unpredictable
phenomenon.
All possible outcomes of the random experiment are collected in a set Ω, which is called the
sample
space
. Examples:
1. A die is thrown : possible outcomes are “1 dot”, “2 dots”, ..., “6 dots”. Then Ω =
{
1 dot
,
2 dots
, . . . ,
6 dots
}
2. A coin is tossed twice. Then Ω =
{
HH, HT, T H, T T
}
.
3. System identification experiment described in class: Ω consists of all possible values of the paur
(ˆ
a
d
,
ˆ
b
d
).
An event is a collection of such outcomes. Examples:
1. (in the die toss experiment)
E
Δ
=
{
1
,
2
,
4
,
6
}
. This is the event that an even number of dots turn
up.
2. (in the system identification experiment)
E
Δ
=
{
ˆ
a
d
<
1
}
. (What’s this event?)
In general, and event
E
is a subset of the sample space, i.e.,
E
⊂
Ω. Based on the sample space Ω, a
larger set
F
is defined:
F
=
{
E

E
⊆
Ω
, E
is “measurable”
}
which is called a
σ
field or
σ
algebra
. We will not go into what a measurable set is. For the purpose
of this course, you can take it that all sets are measurable, so that every subset of Ω is an element of
the
σ
algebra
F
. Think of
F
as the set of all events.
F
contains Ω and
φ
, the sample space and the
empty set, among other events. When Ω is thought of as an event (an element of
F
), it is called the
sure event.
φ
is called the impossible event.
There are several ways of thinking of probability.
*
Please email me at [email protected] if you find any typos.
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
1.
Probability as the relative frequency of occurrence
: do the underlying random experiment
a very large number of times. Probability of an event is the ratio between the number of times
the event occurs to the total number of experiments.
2.
Probability as the ratio of favorable to all possible outcomes:
Example: say the event
E
= an even number of dots appear when a die is thrown. The favorable outcomes, for which
the event is said to occur, are 2
,
4
,
6, whereas all the possible outcomes are 1
, . . . ,
6.
Then,
probability of
E
is 3
/
6, i.e., 1
/
2. Note that all the outcomes are assumed to be equally likely for
this calculation to make sense.
3.
Modern, or axiomatic, definition of probability:
A probability
P
is a function that assigns
a number between 0 and 1 to every event, i.e., to every element of the
σ
field
F
.
Therefore,
probability is a function
P
:
F
→
[0
,
1]
that satisfies the following axioms:
•
P
(
E
)
≥
0 for every
E
∈
F
.
•
P
(Ω) = 1.
•
If
A
∩
B
=
φ
, then
P
(
A
∪
B
) =
P
(
A
) +
P
(
B
)
The triplet (Ω
, F, P
) is called a
probability space
.
Two events
A
and
B
are called
mutually
exclusive
if
A
∩
B
=
φ
.
1.1.1
Operations on sets
A
∩
B
Δ
=
{
ω
∈
Ω

ω
∈
A, ω
∈
B
}
= “
A
and
B
” (both the events A and B occur)
A
∪
B
Δ
=
{
ω
∈
Ω

ω
∈
A
or
ω
∈
B
}
= “
A
or
B
” (either A occurs, or B occurs, or both occur)
A
c
Δ
=
{
ω
∈
Ω

ω /
∈
A
}
= “not A” (A does not occur)
A

B
Δ
=
{
ω
∈
Ω

ω
∈
A, ω /
∈
B
}
= “
A
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '08
 Staff
 Probability distribution, Probability theory, probability density function

Click to edit the document details