This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Introduction to Time Series Analysis. Lecture 13.
Peter Bartlett Last lecture: 1. YuleWalker estimation 2. Maximum likelihood estimator 1 Introduction to Time Series Analysis. Lecture 13.
1. Review: Maximum likelihood estimation 2. Computational simplifications: un/conditional least squares 3. Diagnostics 4. Model selection 5. Integrated ARMA models 2 Review: Maximum likelihood estimator
Suppose that X1 , X2 , . . . , Xn is drawn from a zero mean Gaussian ARMA(p,q) process. The likelihood of parameters Rp , Rq , 2 w R+ is defined as the density of X = (X1 , X2 , . . . , Xn ) under the Gaussian model with those parameters:
2 L(, , w ) = 1 (2)n/2 n 1/2 1 exp  X 1 X , n 2 where A denotes the determinant of a matrix A, and n is the variance/covariance matrix of X with the given parameter values.
2 The maximum likelihood estimator (MLE) of , , w maximizes this quantity. 3 Maximum likelihood estimation: Simplifications
We can simplify the likelihood by expressing it in terms of the innovations. Since the innovations are linear in previous and current values, we can write 0 X1 X1  X1 . . . . =C . . n1 Xn Xn  Xn
X U where C is a lower triangular matrix with ones on the diagonal. Take the variance/covariance of both sides to see that n = CDC
0 n1 where D = diag(P1 , . . . , Pn ). 4 Maximum likelihood estimation
0 n1 0 n1 Thus, n  = C2 P1 Pn = P1 Pn and X 1 X = U C 1 CU = U C C T D 1 C 1 CU = U D 1 U. n n So we can rewrite the likelihood as
2 L(, , w ) = n i1 (Xi  Xi )2 /Pii1 i=1 1
n1 0 (2)n P1 Pn 1/2 1 exp  2 = 1
0 2 (2w )n r1 n1 1/2 rn S(, ) exp  2 2w , i1 2 where ri = Pii1 /w and n S(, ) =
i=1 i1 Xi  Xi i1 ri 2 . 5 Maximum likelihood estimation
2 The log likelihood of , , w is 2 2 l(, , w ) = log(L(, , w )) n 1 2 =  log(2w )  2 2 n i1 log ri  i=1 S(, ) . 2 2w 2 ^ ^ ^2 Differentiating with respect to w shows that the MLE (, , w ) satisfies ^ ^ S(, ) n = 2 2^w 2^w 4 ^ ^ and , minimize log ^ ^ S(, ) n w ^2 ^ ^ S(, ) , = n
n i1 log ri . i=1 1 + n 6 Summary: Maximum likelihood estimation
^ ^ ^2 The MLE (, , w ) satisfies w ^2 ^ ^ and , minimize
i1 2 where ri = Pii1 /w and n i1 Xi  Xi i1 ri 2 ^ ^ S(, ) , = n ^ ^ S(, ) log n 1 + n n i1 log ri , i=1 S(, ) =
i=1 . 7 Maximum likelihood estimation
Minimization is done numerically (e.g., NewtonRaphson). Computational simplifications: i1 Unconditional least squares. Drop the log ri terms. Conditional least squares. Also approximate the computation of xi1 by i dropping initial terms in S. e.g., for AR(2), all but the first two terms in S depend linearly on 1 , 2 , so we have a least squares problem. The differences diminish as sample size increases. For example, i1 t1 2 Ptt1 w so rt 1, and thus n1 i log ri 0. 8 Review: Maximum likelihood estimation
For an ARMA(p,q) process, the MLE and un/conditional least squares estimators satisfy 1 2 ^ w  AN 0, , ^ n , = Cov((X, Y ), (X, Y )), where , X = (X1 , . . . , Xp ) Y = (Y1 , . . . , Yp ) (B)Xt = Wt , (B)Yt = Wt . 9 Introduction to Time Series Analysis. Lecture 13.
1. Review: Maximum likelihood estimation 2. Computational simplifications: un/conditional least squares 3. Diagnostics 4. Model selection 5. Integrated ARMA models 10 Building ARMA models
1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Nonlinearly transform data, if necessary 3. Identify preliminary values of p, and q. 4. Estimate parameters. 5. Use diagnostics to confirm residuals are white/iid/normal. 6. Model selection: Choose p and q. 11 Diagnostics
How do we check that a model fits well? The residuals (innovations, xt  xt1 ) should be white. t Consider the standardized innovations, et = xt  xt1 ^t ^ Ptt1 . This should behave like a meanzero, unit variance, iid sequence. Check a time plot Turning point test Difference sign test Rank test QQ plot, histogram, to assess normality
12 Testing i.i.d.: Turning point test
{Xt } i.i.d. implies that Xt , Xt+1 and Xt+2 are equally likely to occur in any of six possible orders:
3.5 3 2.5 2 1.5 1 0.5 0 5 10 15 20 (provided Xt , Xt+1 , Xt+2 are distinct). Four of the six are turning points. 13 Testing i.i.d.: Turning point test
Define T = {t : Xt , Xt+1 , Xt+2 is a turning point}. ET = (n  2)2/3. Can show T AN (2n/3, 8n/45). Reject (at 5% level) the hypothesis that the series is i.i.d. if 2n > 1.96 T 3 8n . 45 Tests for positive/negative correlations at lag 1. 14 Testing i.i.d.: Differencesign test
S = {i : Xi > Xi1 } = {i : ( X)i > 0}. ES = n1 . 2 Can show S AN (n/2, n/12). Reject (at 5% level) the hypothesis that the series is i.i.d. if S Tests for trend.
(But a periodic sequence can pass this test...) n > 1.96 2 n . 12 15 Testing i.i.d.: Rank test
N = {(i, j) : Xi > Xj and i > j}. n(n  1) EN = . 4 Can show N AN (n2 /4, n3 /36). Reject (at 5% level) the hypothesis that the series is i.i.d. if n2 > 1.96 N 4 Tests for linear trend. n3 . 36 16 Testing if an i.i.d. sequence is Gaussian: qq plot
Plot the pairs (m1 , X(1) ), . . . , (mn , X(n) ), where mj = EZ(j) , Z(1) < < Z(n) are order statistics from N (0, 1) sample of size n, and X(1) < < X(n) are order statistics of the series X1 , . . . , Xn . Idea: If Xi N (, 2 ), then EX(j) = + mj , so (mj , X(j) ) should be linear. There are tests based on how far correlation of (mj , X(j) ) is from 1. 17 Introduction to Time Series Analysis. Lecture 13.
1. Review: Maximum likelihood estimation 2. Computational simplifications: un/conditional least squares 3. Diagnostics 4. Model selection 5. Integrated ARMA models 18 ...
View
Full
Document
This note was uploaded on 05/14/2011 for the course STAT 153 taught by Professor Staff during the Fall '08 term at University of California, Berkeley.
 Fall '08
 Staff

Click to edit the document details