15 / 32
Properties of
b
γ
and
b
ρ
b
γ
n
and
b
ρ
are biased but consistent as
n
→ ∞
under certain conditions.
Theorem 3 (CLT of
b
ρ
)
If
{
X
t
}
is the stationary process
X
t

μ
=
∞
X
j
=
∞
ψ
j
Z
t

j
,
{
Z
t
} ∼
IID
(0
, σ
2
)
,
where
∞
∑
j
=
∞

ψ
j

<
∞
and
E
Z
4
<
∞
, then for each
h
, we have
√
n
(
b
ρ
(
h
)

ρ
(
h
)) =
⇒ N
(0
, W
h
)
,
where
b
ρ
(
h
) = [
b
ρ
(1)
,
· · ·
,
b
ρ
(
h
)]
T
and
ρ
(
h
) = [
ρ
(1)
,
· · ·
, ρ
(
h
)]
T
.
W
h
is a
nonnegative definition matirx (see Theorem 7.2.1 of B & D for the expression of
W
).
16 / 32
Estimation of ARMA Models
17 / 32
ARMA Modeling
The determination of an appropriate ARMA(p,q) model to represent an observed
stationary time series involves a number of interrelated problems.
The choice of
p
and
q
(Order Selection);
Estimation of ARMA coefficients and the white noise variance, given
p
and
q
;
Check goodness of fit.
Throughout we assume the data has been adjusted by subtraction of the mean.
Therefore, we only consider zeromean ARMA models. The modeling procedure
often starts with plotting the estimated ACF and PACF of the data.
If ACF decreases to
0
quickly, the process is suspected to be a MA process.
If PACF decreases to
0
quickly, the process is suspected to be an AR process.
First obtain preliminary estimators of
φ
’s and/or
θ
’s.
More efficient esimation:
maximum likelihood estimators
. Need preliminary
estimators as initial values for optimization.
18 / 32
Preliminary Estimation of AR(p)
Preliminary estimation of AR coefficients and PACF estimators can be obtained by
the
YuleWalker Equation
and DurbinLevinson Algorithm.
Let
{
X
t
}
be the zeromean causal autoregressive process
X
t
=
φ
1
X
t

1
+
· · ·
+
φ
p
X
t

p
+
Z
t
,
{
Z
t
} ∼
WN
(0
, σ
2
)
.
Our aim is to find estimators of the coefficient vector
φ
= (
φ
1
, . . . , φ
p
)
T
and the
white noise variance
σ
2
based on the observations
X
1
, . . . , X
n
. Multiplying both
sides by
X
t

j
,
j
= 0
, . . . , p
and taking expectations, we obtain the YuleWalker
equations,
Γ
p
φ
=
γ
p
,
and
σ
2
=
γ
(0)

φ
T
γ
p
.
We replace the covariance
γ
(
j
)
by the corresponding sample covariances
b
γ
(
j
)
, we
obtain a set of equations for the socalled YuleWalker estimators
b
φ
and
b
σ
2
of
φ
and
σ
2
, namely
b
Γ
p
b
φ
=
b
γ
p
and
b
σ
2
=
b
γ
(0)

b
φ
T
b
γ
p
.
19 / 32
Distribution of
b
φ
Theorem 4
If
{
X
t
}
is the causal AR(p) process with
{
Z
t
} ∼
IID
(0
, σ
2
)
, and
b
φ
is the
YuleWalker estimator of
φ
, then
n
1
/
2
(
b
φ

φ
) =
⇒ N
(0
, σ
2
Γ

1
p
)
,
Moreover,
b
σ
2
P
→
σ
2
.
20 / 32
Order Misspecification
If the true order is
p
and we attempt to fit a process of order
m
for
m
≤
n

1
, we
should expect the estimated coefficient vector
φ
m
= (
b
φ
m
1
,
· · ·
,
b
φ
mm
)
T
to have a
small value of
b
φ
mm
for each
m > p
, since
b
φ
mm
is the estimator of the PACF at
lag
m
. The following theorem is extremely useful in helping us to identify the
appropriate order of the process to be fitted.
You've reached the end of your free preview.
Want to read all 34 pages?
 Fall '14