Chapter 6: Parameter Estimation and
Evaluation
References:
Chapter 9, Sections 9.1°9.13 (only for references)
6.1 Point estimation
Suppose
f
X
i
g
n
i
=1
is an i.i.d. random sample from some population
f
X
(
x
)
:
[Recall the popu-
lation distribution
f
X
(
x
)
is the distribution of each
X
i
:
]
Often, one assumes
f
X
(
x
) =
f
(
x; °
)
;
where
f
(
x; °
)
is unknown in functional form, but
°
is unknown.
For example, if
f
X
i
g
is
i.i.d.
N
(
±; ²
2
)
;
then
f
X
(
x
)
=
f
(
x; °
)
=
1
p
2
³²
2
exp[
°
1
2
²
2
(
x
°
±
)
2
]
;
where
°
= (
±; ²
2
)
:
Suppose the random sample
X
n
=
f
X
1
; X
2
; ::; X
n
g
has a realization
x
n
=
f
x
1
; x
2
; :::; x
n
g
:
Then the realization
x
n
is called a data set. A random sample
X
n
can generate many di/erent
data sets.
Estimator versus Estimate
De°nition [Estimator and Estimate]:
Suppose
°
is a parameter of a population, and
is unknown.
An estimator
^
°
=
^
°
(
X
1
; :::; X
n
)
of
°
is a statistic that depends on the sample
information and whose realizations provide an approximation to
°:
A speci±c realization of that
r.v. is called an estimate.
De°nition [Point Estimator]:
A point estimator
^
°
is an estimator for
°
that only gives a
single number for each sample realization
x
n
= (
x
1
; x
2
; :::; x
n
)
.
Example: Suppose
X
n
= (
X
1
; :::; X
n
)
is a random sample with
(
±; ²
2
)
:
Then we have
°
=
±;
^
°
=
°
X
n
=
n
°
1
P
n
i
=1
X
i
:
Example: Suppose
X
n
= (
X
1
; :::; X
n
)
is a random sample where
X
i
±
Bernoulli
(
p
)
;
0
< p <
1
:
Then
°
=
p;
^
°
= ^
p
n
=
°
X
n
:
Example: Suppose
X
n
= (
X
1
; :::; X
n
)
is a random sample with
(
±; ²
2
)
:
Then we have
°
=
²
2
;
^
°
=
S
2
n
= (
n
°
1)
°
1
P
n
i
=1
(
X
i
°
°
X
n
)
2
:
1
This
preview
has intentionally blurred sections.
Sign up to view the full version.
Example: Suppose
X
n
= (
X
1
; :::; X
n
)
is a random sample with
(
±
1
; ²
2
1
)
;
and
Y
m
= (
Y
1
; :::; Y
m
)
is a random sample with
(
±
2
; ²
2
2
)
:
Then we have
°
=
±
1
°
±
2
;
and
^
°
=
°
X
n
°
°
Y
m
:
Example: Suppose
X
n
= (
X
1
; :::; X
n
)
is a random sample with
(
±
1
; ²
2
1
)
;
and
Y
m
= (
Y
1
; :::; Y
m
)
is a random sample with
(
±
2
; ²
2
2
)
:
Then
°
=
²
2
1
=²
2
2
and
^
°
=
S
2
n
=S
2
m
:
Notations of Estimators and Estimates:
parameter
(
°
)
Estimator
(
^
°
)
Estimate
±
°
X
n
°
x
n
²
2
S
2
n
s
2
n
²
S
n
s
n
p
^
p
n
p
n
:
Example:
Price-earnings ratios for a random sample of ten stocks traded in NYSE on a partic-
ular date is
10
;
16
;
5
;
10
;
12
;
8
;
4
;
6
;
5
;
4
:
Find the point estimates of the population mean, variance and standard deviation, and the
proportion of stocks in the population for which the price-earnings ratio exceeded 8.5.
ANS:
±; ²
2
; ²
and
p
=
P
(
X >
8
:
5)
:
°
x
n
= 80
=
10 = 8
:
s
2
x
= (
n
°
1)
°
1
(
P
n
i
=1
x
2
°
n
°
x
2
n
) = (782
°
10
²
8
2
)
=
9 = 15
:
78
:
s
x
=
p
15
:
78 = 3
:
97
:
^
p
x
= 4
=
10 = 0
:
4
:
Questions: For any
°
, ther are more than one estimators.
Example: Let
°
=
±;
then the following are estimators for
°
=
±
:
(i)
^
°
1
=
°
X
n
:
(ii)
^
°
2
=
X
1
:
Which one is a better estimator?
How good is an estimator
^
°
for
°
?
We need to use some appropriate criterion to measure how
good the estimator is.
De°nition [Mean Squared Error (MSE)]:
The MSE of an estimator
^
°
is de±ned as
MSE
(
^
°
) =
E
(
^
°
°
°
)
2
:
Remarks: MSE(
^
°
)
measures the variations or deviations of the estimator
^
°
from the true
parameter
°
. The smaller MSE(
^
°
)
;
the better the estimator
^
°:
A smaller MSE means that there
is smaller variation between
^
°
and
°:
De°nition [Relative E¢ ciency]:

This is the end of the preview.
Sign up
to
access the rest of the document.
- Fall '10
- Estimation theory, Mean squared error, Bias of an estimator, M SE
-
Click to edit the document details