EE7615
A Review of Decision and Estimation Theory
If
M
is the outcome of some random experiment we would like to know but can not
observe, and if
R
is the outcome of some random experiment we can observe, and if
M
and
R
are not independent, then it is reasonable to expect that
R
might profitably be used to
make an estimate of the unobserved value of
M
. The study of rules for doing just this is
the topic of decision and estimation theory. This theory tells us how to find a decision (or
estimation) rule which, for any
R
=
r
that might occur, indicates a good guess for
M
. That
is, an estimation rule is a function which assigns an estimate ˆ
m
to every potential observed
value of
R
=
r
. In particular the theory can provide us with the best such rule.
In order to characterize the best rule we must know the relationship between
M
and
R
,
i.e., their joint distribution. In addition we must specify how to measure the ”goodness” of
an estimation rule. Indeed, for different measures of goodness different rules turn out to be
the best.
Suppose we have two random variables (r.v.)
M
and
R
.
M
is discrete with alphabet
Ω
M
=
{
m
1
, m
2
, ..., m
q
}
, while
R
can be discrete or continuous and has alphabet Ω
R
. The
joint distribution of
M
and
R
is given by
p
MR
(
m, r
). If
R
is discrete then
p
MR
(
m, r
) is the
probability mass function (pmf), while if
R
is continuous,
p
MR
(
m, r
) is mixed. We want to
know the value of
M
. But we can only observe the outcome of the r.v.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '11
 Naragipour
 Maximum likelihood, Estimation theory

Click to edit the document details