Probability and Statistics
Cumulative Distribution Function (CDF)
of a Random Variable (RV) X
CDF
X
(
z
) = Pr (
X
°
z
)
Discrete:
P
n
i
=1
1 (
x
i
°
z
) Pr (
X
=
x
i
)
;
where
1 (
E
) = 1
if
E
is true, and 0 otherwise.
Continuous:
R
z
°1
f
(
x
)
dx
=
R
z
°1
1 (
x
°
z
)
f
(
x
)
dx
Expectation
of
X
Discrete Random Variable:
°
X
=
E
(
X
) =
x
1
Pr(
X
=
x
1
) +
:::
+
x
n
Pr(
X
=
x
n
) =
P
n
i
=1
x
i
Pr(
X
=
x
i
)
Continuous Random Variable:
°
X
=
E
(
X
) =
R
1
°1
xf
(
x
)
dx
Expectation of a function
of
X
E
[
g
(
X
)] =
P
n
i
=1
g
(
x
i
) Pr(
X
=
x
i
)
or
E
[
g
(
X
)] =
R
1
°1
g
(
x
)
f
(
x
)
dx
Variance
V ar
(
X
) =
E
h
(
X
±
°
X
)
2
i
=
±
2
X
V ar
(
X
) =
P
n
i
=1
[
x
i
±
°
X
]
2
Pr(
X
=
x
i
)
or
V ar
(
X
) =
R
1
°1
(
x
±
°
X
)
2
f
(
x
)
dx
V ar
(
X
) =
E
(
X
2
)
±
(
°
X
)
2
r
th
moment
E
(
X
r
) =
P
n
i
=1
x
r
i
Pr(
X
=
x
n
)
;
or
E
(
X
r
) =
R
1
°1
x
r
f
(
x
)
dx
Marginal Probability
P
(
Y
=
y
) =
n
P
i
=1
P
(
X
=
x
i
; Y
=
y
)
and Marginal Density
R
1
°1
f
(
y; x
)
dx
Conditional Probability
P
(
Y
=
y
j
X
=
x
) =
P
(
X
=
x;Y
=
y
)
P
(
X
=
x
)
and Conditional Density
f
(
y;x
)
f
(
x
)
Conditional Expectation
E
[
Y
j
X
=
x
] =
P
n
i
=1
y
i
P
(
Y
=
y
i
j
X
=
x
)
or
R
1
°1
yf
(
y
j
X
=
x
)
dy
Law of Iterated Expectations
E
[
E
(
Y
j
X
)] =
E
[
Y
]
De°nitions of Independence
P
(
Y
=
y
j
X
=
x
) =
P
(
Y
=
y
)
or
P
(
X
=
x; Y
=
y
) =
P
(
X
=
x
)
P
(
Y
=
y
)
Covariance and Correlation
cov
(
X; Y
) =
E
[(
X
±
°
X
)(
Y
±
°
Y
)] =
±
XY
cov
(
X; Y
) =
P
n
i
=1
P
m
j
=1
(
x
i
±
°
X
)(
y
j
±
°
Y
)
P
(
X
=
x
i
; Y
=
y
j
)
cov
(
X; Y
) =
R
1
°1
R
1
°1
(
x
±
°
X
)(
y
±
°
Y
)
f
(
x; y
)
dxdy
²
=
cov
(
X;Y
)
°
X
°
Y
Means, Variances and Covariances of Sums of Random Variables
E
(
a
+
bX
+
cY
) =
a
+
b°
X
+
c°
Y
var
(
a
+
bX
+
cY
) =
b
2
±
2
X
+ 2
bc±
XY
+
c
2
±
2
Y
cov
(
a
+
bX
+
cY; d
+
eZ
+
fW
) =
be±
XZ
+
bf±
XW
+
ce±
Y Z
+
cf±
Y W
E
(
XY
) =
±
XY
+
°
X
°
Y
Some useful estimators
Sample Mean:
°
X
=
1
n
P
n
i
=1
X
i
:
Sample Variance:
s
2
X
=
1
n
°
1
P
n
i
=1
°
X
i
±
°
X
±
2
:
Sample Covariance:
s
XY
=
1
n
°
1
P
n
i
=1
(
X
i
±
°
X
)(
Y
i
±
Y
)
note that
P
n
i
=1
(
X
i
±
°
X
)(
Y
i
±
Y
) =
P
n
i
=1
(
X
i
±
°
X
)
Y
i
=
P
n
i
=1
X
i
(
Y
i
±
Y
)
Sample Correlation:
r
XY
=
s
XY
s
X
s
Y
Independent and Identically Distributed
(
iid
)
random variable.
X
1
; X
2
; :::; X
n
are
iid
random variables if
E
(
X
i
) =
°
X
and
V ar
(
X
i
) =
±
2
X
8
(that is, for all)
i
then we can write
X
i
²
iid
°
°
X
; ±
2
X
±
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
³
Convergence in Probability.
Loosely speaking, we say that the random variable
S
n
converges in probability
to
a;
or
S
n
p
!
a
or, equivalently,
plim
n
!1
S
n
=
a
if
S
n
becomes °closer and closer±to
a
as
n
! 1
:
Formally,
S
n
converges in probability to
a
if for every
" >
0
Pr (
j
S
n
±
a
j
> "
)
!
0
as
n
! 1
³
We say that
^
³
n
(or, for brevity,
^
³
) is a
consistent estimator
for a parameter
³
if the estimator
^
³
n
converges
in probability to
³
when
n
! 1
;
that is,
^
³
n
p
!
³
or
plim
n
!1
^
³
n
=
³:
³
Law of Large Numbers.
Let
X
1
; X
2
; ::; X
n
be
iid
random variables with mean
°
X
and
var
(
X
i
) =
±
2
X
<
1
:
Then
X
is a consistent estimator of
°
X
;
that is
°
X
p
!
°
X
:
³
Convergence in Distribution
. We say that
S
n
converges in distribution to
S
, and we write
S
n
d
!
S;
if
distribution
of
S
n
becomes °close±to the
distribution
of
S
as
n
! 1
³
Central Limit Theorem.
If
X
1
; :::; X
n
are
iid
°
°
X
; ±
2
X
±
with
0
< ±
2
X
<
1
, then
X
±
°
X
°
X
p
n
=
p
n
X
±
°
X
±
X
d
!
N
(0
;
1)
Note that this also implies that
p
n
°
°
X
±
°
X
±
d
!
N
°
0
; ±
2
X
±
³
Slutsky±s Theorem.
Suppose that
a
n
p
!
a
and
S
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '08
 ALESSANDROTAROZZI
 Econometrics, Linear Regression, Regression Analysis, Yi

Click to edit the document details