443: STAT Forecasting
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Forecasting
STAT 443: Forecasting
Forcasting
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
More on one step forecasting for ARMA models h-step ahead forecasting for ARMA models Forecasting with ARIMA models Relationship between Holt-Winters and ARIMA
modelling
STAT 443: Forecasting
Forecasting
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Have algorithms for computing P(Xn+1 |Xn , . . . , X1 )
based on ACVF (h)
For AR(p) use Yule-Walker or Durbin-Levinson For MA(q) and ARMA(p, q) use innovation algorthim With data replace ACVF (h) in algorithm with sample
ACVF (h)
STAT 443: Forecasting
Forcasting ARMA(p, q)
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
For an ARMA(p, q) process (B)Xt = (B)Zt where
Zt WN(, 2 ) and let m = max(p, q) Then P(Xn+1 |Xn , . . . , X1 ) is given by
bnj (Xn+1-j - Xn+1-j ) 1 Xn + + n Xn+1-p +
n j=1 q j=1
bnj (Xn+1-j
1n<m - Xn+1-j ),n m
where Xn+1 = P(Xn+1 |Xn , . . . , X1 ) and bij are
determined by the innovation algorithm using the sample ACVF function
STAT 443: Forecasting
Forcasting AR(p)
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Using this in the case of AR(p) model (B)Xt = Zt get P(Xn+1 |Xn , . . . , X1 ) is given by
1 Xn + + n Xn+1-p when n > p.
So Yule-Waker or Durbin-Levinson can be used with
sample ACVF
STAT 443: Forecasting
Forcasting MA(q)
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Using this in the case of MA(q) model Xt = (B)Zt get P(Xn+1 |Xn , . . . , X1 ) is given by
min {n,q}
bnj (Xn+1-j - Xn+1-j )
j=1
when n 1.
where bij are determined by the innovation algorithm
STAT 443: Forecasting
Forcasting ARMA(1, 1)
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Using the model Xt - Xt-1 = Zt + Zt-1 get P(Xn+1 |Xn , . . . , X1 ) is given by
Xn + bn1 (Xn - Xn ) when n 1.
where bij are determined by the innovation algorithm
STAT 443: Forecasting
h-step forecasting
Notation
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Pn Xn+h = P(Xn+h |Xn , , X1 ) the h-step ahead predictor
Can write explicitly as
n
Pn Xn+h = +
i=1
ai (Xn+1-i - )
where a = (a1 , . . . , an )T satisfies n a = n (h) = ((h), (h + 1), . . . , (h + n - 1))T
STAT 443: Forecasting
Properties of Pn Xn+h
1
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
MSE is E (Xn+h - Pn Xn+h )2 = (0) - aT n (h)
2
Pn Xn+h is unbiased estimate of mean of Xn+h i.e. E[Xn+h - Pn Xn+h ] = 0
3
Error is uncorrelated with predictor, i.e. E[{Xn+h - Pn Xn+h } Xj ] = 0 for j = 1, , n
STAT 443: Forecasting
Characterising Pn Xn+h
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Properties (2) and (3) are equivalent to
n
E(Xn+h - a0 -
i=1 n
ai Xn+1-i ) = 0 Xn+1-j = 0
E
Xn+h - a0 -
i=1
ai Xn+1-i
for j = 1, . . . , n
These sets of equations characterise Pn Xn+h
STAT 443: Forecasting
Recursive calculation of Pn Xn+h
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Prediction identity
Pn (Xn+k - Pn+k-1 Xn+k ) = 0 for k 1
i.e. the best forecast (based on Xn , . . . , X1 ) for the
innovation at n + k + 1 is its mean, 0
STAT 443: Forecasting
Proof of Identity
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Let Un,k = Xn+k - Pn+k-1 Xn+k be the error For k 1 and j = 1, . . . , n have by Property (3)
E - (Un,k 0)Xn-j-1 = 0
Since the equations characterise the predictor of Un,k it
follows that this predictor is 0.
Giving the result
Pn Un,k = Pn (Xn+k - Pn+k-1 Xn+k ) = 0
STAT 443: Forecasting
Characterising Pn Xn+h
The identity gives that
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Pn Xn+h = Pn Pn+h-1 Xn+h
n+h-1
= Pn
j=1
bn+h-1,j (Xn+h-j - Pn+h-j-1 Xn+h-j )
Using the identity we see that the terms for
j = 1, . . . , h - 1 are zero, and Pn is the identity for the remaining terms
This gives
n+h-1
Pn Xn+h =
j=h
bn+h-1,j (Xn+h-j - Pn+h-j-1 Xn+h-j )
STAT 443: Forecasting
Recursive calculation of Pn Xn+h
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
The terms bn+h-1,j can be computed using the
innovations algorithm
The MSE is given by
n+h-1
(n + h, n + h) -
j=h
2 bn+h-1,j n+h-j-1
STAT 443: Forecasting
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Forecasting non-stationary time series
In order to extend forecasting to the non-stationary time
series some further assumptions about initial conditions are needed
Recall d 1 {Xt } is ARIMA(p, d, q) if Yt = (1 - B)d Xt
is a causal ARMA(p, q) process
So Xt satisfies
(B)(1 - B)d Xt = (B)Zt Zt WN(0, 2 )
Consider the d = 1 case
STAT 443: Forecasting
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Forecasting non-stationary time series
Consider the case
t
Xt = X0 +
j=1
Yj
Then {Xt } is an ARIMA(p, 1, q) process with mean and
autocovariance E(Xt ) = E(X0 ), E(Xt+h Xt ) - (E(X0 ))2
Have
Pn Xn+1 = Pn (X0 + Y1 + + Yn+1 ) = Pn (Xn + Yn+1 ) = Xn + Pn Yn+1
STAT 443: Forecasting
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Forecasting non-stationary time series
To be able to apply the ARMA(p, q) formula for Pn Yn+1
need to assume that X0 has now effect on prediction
If we assume that X0 is uncorrelated with Y1 , Y2 ,
then the prediction of Yn given X0 is equal to that from the ARMA process.
When d > 1 need further assumptions on initial
conditions X0 , X1 , . . .
STAT 443: Forecasting
Exponential Smoothing
Suppose we have a trend plus noise model
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Yn = mn + Zn , where E(Zn ) = 0
Exponential smoothing is defined by
mn = Yn + (1 - )mn-1 where [0, 1] and m1 = Y1 .
Equivalently write update rule in error correction form
mn = mn-1 + (Yn - mn-1 ) i.e. update estimate with last one step forecast error
STAT 443: Forecasting
Exponential Smoothing
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
In different notation can write exponential smoothing as
for Yn as Pn Yn+1 = Pn-1 Yn + (Yn - Pn-1 Yn ) = Yn - (1 - )(Yn - Pn-1 Yn )
Now suppose that Yn ARIMA(0, 1, 1) Define Xn+1 = Yn+1 - Yn this is a MA(1) process of the
form Xn+1 = Zn+1 - Zn
STAT 443: Forecasting
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
Exponential Smoothing and ARIMA
Use the fact that for an MA(1) process have
Pn Xn+1 = bn (Xn - Xn )
For large samples have that bn Gives
Pn Yn+1 - Yn = Pn Xn+1 = (Xn - Pn-1 Xn ) = (Yn - Yn-1 - Pn-1 (Yn - Yn-1 )) = (Yn - Pn-1 Yn )
Same as Exponential smoothing with = 1 - .
STAT 443: Forecasting
Holt-Winters and ARIMA
ARMA Forecasting h-step forecasting Forecasting ARIMA(p, d.q) models ARIMA and Holt-Winters
So exponential smoothing is optimal when the model is
ARIMA(0, 1, 1) in large samples
It can be shown that Holt-Winters forecasting can be
viewed as fitting (1 - B)2 Yt = Zt - (2 - - )Zt-1 + (1 - )Zt-2
So is optimal when data is ARIMA(0, 2, 2)