Chapter 1
Probability Models
Broadly speaking, there are two types of models.
1.
Parametric models
: A family of distributions
F
=
{
F
θ
:
θ
∈
Θ
}
is said to be a
parametric family
iff Θ
⊂
R
d
for some fixed positive
integer
d
, and each
F
θ
is known when
θ
is known. The set Θ is called
the
parameter space
and
d
is called its
dimension
.
(Note that the form of distributions is known except for some fixed num
ber of unknown parameters.)
A parametric model
refers to the assumption that the population dis
tribition is in a parametric family. A parametric model with very high
dimension
d
is not very useful.
A parametric family is
identifiable
iff
θ
1
6
=
θ
2
implies that
F
θ
1
6
=
F
θ
2
.
(If we know a sample is from
F
θ
, then we can uniquely determine (or
identify)
θ
. Therefore, we can talk about estimating
θ
.)
e.g. Normal family:
F
=
{
N
(
μ, σ
2
) :
μ
∈
R, σ
2
>
0
}
. Here
θ
= (
μ, σ
2
).
2.
Nonparametric models
: not parametric models. Here, the form
of distributions cannot be decided by a fixed number of unknown pa
rameters, hence it is sometimes called an infinite dimensional problem.
In nonparametric models, minimal assumptions are usually made re
garding the form of distributions, e.g.,
(a) All continuous d.f.’s.
(b) All symmetric d.f.’s.
(c) All d.f. with finite moments of order
r >
0.
(d) All d.f. with p.d.f.’s.
(e) All d.f. continuous and symmetric.
There are also so called
Semiparametric models
, which are mixtures of
parametric and nonparametric models (such as Cox’s model). They could be thought
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
to include parametric and nonparametric models as special cases. On the other hand,
they can also be regarded as nonparametric models.
In this course, we shall mostly concentrate on
parametric models
. Two very
useful parametric models will be introduced below:
(a). exponential family;
(b). locationscale family
.
1.1
Exponential family
Definition
1.1 (Exponential family)
A family
F
=
{
F
θ
:
θ
∈
Θ
}
of distri
butions is said to be a
k
parameter exponential family if the distributions
F
θ
have
densities of the form (either with Lesbegue or counting measures)
f
θ
(
x
) = exp
k
X
j
=1
η
j
(
θ
)
T
j
(
x
)

B
(
θ
)
h
(
x
)
≡
exp
k
X
j
=1
η
j
(
θ
)
T
j
(
x
)
C
(
θ
)
h
(
x
)
,
with respect to some common measure
μ
.
Here all the functions are realvalued,
h
(
x
)
≥
0
.
Remark
1.1
•
Since
R
f
θ
(
x
)
d
x
= 1
, we find that
B
(
θ
) = ln
Z
exp
k
X
j
=1
η
j
(
θ
)
T
j
(
x
)
h
(
x
)
d
x
.
Therefore,
η
j
(
θ
)
’s are “free” parameters, while
B
(
θ
)
depends on
η
j
’s and hence
not a free “parameter”.
•
Many common families are exponential families.
These include the normal,
gamma, beta distributions, and binomial, negative binomial etc. The exponen
tial families have many nice properties, as can be seen below.
•
The above representation is not unique, since
η
j
T
j
= (
cη
j
)(
T
j
/c
)
.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '11
 D
 Normal Distribution, Probability, exponential family, Sufficient statistic, Factorization Theorem, minimal sufficient statistic

Click to edit the document details