Statistics 3858 : Statistical Models, Parameter Space and
Identifiability
In an experiment or observational study we have data
X
1
, X
2
, . . . , X
n
. These we view as a observations
of random variables with some joint distribution.
Definition 1
A statistical model is a family of distributions
F
such that for any possible
n
, a given
distribution
f
∈ F
gives a joint distribution of
X
1
, X
2
, . . . , X
n
.
Note that
f
above may be either a joint pdf, pmf or cdf. Every
f
∈ F
must specify the (joint) distribution
of
X
i
,
i
1 =
, . . . , n
. Sometimes we use a subscript
n
, that is
f
n
, to indicate the dependence on the sample
size
n
.
For a given sample size
n
, let
f
n
be the joint pdf of the random variables
X
i
, i
= 1
,
2
, . . . , n
. Suppose
the
X
i
’s are iid with marginal pdf
f
. Then the joint pdf is of the form
f
n
(
x
1
, x
2
, . . . , x
n
) =
n
∏
i
=1
f
(
x
i
)
.
(1)
There is of course the analogue for iid discrete r.v.s. Notice also in the iid case the statistical model can
also be viewed or described by the simpler one dimensional marginal distribution.
In this case we can simplify the description of the family
F
to the corresponding family of marginal
distributions
f
. For example if
X
i
’s are iid normal, then the marginal distributions belong to
{
f
(
·
;
θ
) :
θ
= (
µ, σ
2
)
, µ
∈
R, σ
2
∈
R
+
}
.
In many dependent random variables cases we can also obtain their joint distribution. For example
consider the so called autoregressive order one process, AR(1). It is defined iteratively as
X
i
+1
=
βX
i
+
ϵ
i
+1
(2)
Specifically suppose that the r.v.s
ϵ
i
are iid
N
(0
, σ
2
) and independent of the random variables up to time
index less that
i
. Let
f
be the
N
(0
, σ
2
) pdf. Then the conditional distribution of
X
1
given that
X
0
=
x
0
is
f
X
1

X
0
=
x
0
(
x
) =
f
(
x

βx
0
)
Similarly we have the conditional distribution of
X
t
+1
given
X
0
=
x
0
, X
1
=
x
1
, . . . , X
t
=
x
t
, which by
the Markov property is the same as the conditional distribution of
X
t
+1
given
X
t
=
x
t
, given by
f
X
t
+1

X
t
=
x
t
(
x
) =
f
(
x

βx
t
)
1