# 13 - Introduction to Time Series Analysis Lecture 13 Peter...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introduction to Time Series Analysis. Lecture 13. Peter Bartlett Last lecture: 1. Yule-Walker estimation 2. Maximum likelihood estimator 1 Introduction to Time Series Analysis. Lecture 13. 1. Review: Maximum likelihood estimation 2. Computational simplifications: un/conditional least squares 3. Diagnostics 4. Model selection 5. Integrated ARMA models 2 Review: Maximum likelihood estimator Suppose that X1 , X2 , . . . , Xn is drawn from a zero mean Gaussian ARMA(p,q) process. The likelihood of parameters Rp , Rq , 2 w R+ is defined as the density of X = (X1 , X2 , . . . , Xn ) under the Gaussian model with those parameters: 2 L(, , w ) = 1 (2)n/2 |n |1/2 1 exp - X -1 X , n 2 where |A| denotes the determinant of a matrix A, and n is the variance/covariance matrix of X with the given parameter values. 2 The maximum likelihood estimator (MLE) of , , w maximizes this quantity. 3 Maximum likelihood estimation: Simplifications We can simplify the likelihood by expressing it in terms of the innovations. Since the innovations are linear in previous and current values, we can write 0 X1 X1 - X1 . . . . =C . . n-1 Xn Xn - Xn X U where C is a lower triangular matrix with ones on the diagonal. Take the variance/covariance of both sides to see that n = CDC 0 n-1 where D = diag(P1 , . . . , Pn ). 4 Maximum likelihood estimation 0 n-1 0 n-1 Thus, |n | = |C|2 P1 Pn = P1 Pn and X -1 X = U C -1 CU = U C C -T D -1 C -1 CU = U D -1 U. n n So we can rewrite the likelihood as 2 L(, , w ) = n i-1 (Xi - Xi )2 /Pii-1 i=1 1 n-1 0 (2)n P1 Pn 1/2 1 exp - 2 = 1 0 2 (2w )n r1 n-1 1/2 rn S(, ) exp - 2 2w , i-1 2 where ri = Pii-1 /w and n S(, ) = i=1 i-1 Xi - Xi i-1 ri 2 . 5 Maximum likelihood estimation 2 The log likelihood of , , w is 2 2 l(, , w ) = log(L(, , w )) n 1 2 = - log(2w ) - 2 2 n i-1 log ri - i=1 S(, ) . 2 2w 2 ^ ^ ^2 Differentiating with respect to w shows that the MLE (, , w ) satisfies ^ ^ S(, ) n = 2 2^w 2^w 4 ^ ^ and , minimize log ^ ^ S(, ) n w ^2 ^ ^ S(, ) , = n n i-1 log ri . i=1 1 + n 6 Summary: Maximum likelihood estimation ^ ^ ^2 The MLE (, , w ) satisfies w ^2 ^ ^ and , minimize i-1 2 where ri = Pii-1 /w and n i-1 Xi - Xi i-1 ri 2 ^ ^ S(, ) , = n ^ ^ S(, ) log n 1 + n n i-1 log ri , i=1 S(, ) = i=1 . 7 Maximum likelihood estimation Minimization is done numerically (e.g., Newton-Raphson). Computational simplifications: i-1 Unconditional least squares. Drop the log ri terms. Conditional least squares. Also approximate the computation of xi-1 by i dropping initial terms in S. e.g., for AR(2), all but the first two terms in S depend linearly on 1 , 2 , so we have a least squares problem. The differences diminish as sample size increases. For example, i-1 t-1 2 Ptt-1 w so rt 1, and thus n-1 i log ri 0. 8 Review: Maximum likelihood estimation For an ARMA(p,q) process, the MLE and un/conditional least squares estimators satisfy -1 2 ^ w - AN 0, , ^ n , = Cov((X, Y ), (X, Y )), where , X = (X1 , . . . , Xp ) Y = (Y1 , . . . , Yp ) (B)Xt = Wt , (B)Yt = Wt . 9 Introduction to Time Series Analysis. Lecture 13. 1. Review: Maximum likelihood estimation 2. Computational simplifications: un/conditional least squares 3. Diagnostics 4. Model selection 5. Integrated ARMA models 10 Building ARMA models 1. Plot the time series. Look for trends, seasonal components, step changes, outliers. 2. Nonlinearly transform data, if necessary 3. Identify preliminary values of p, and q. 4. Estimate parameters. 5. Use diagnostics to confirm residuals are white/iid/normal. 6. Model selection: Choose p and q. 11 Diagnostics How do we check that a model fits well? The residuals (innovations, xt - xt-1 ) should be white. t Consider the standardized innovations, et = xt - xt-1 ^t ^ Ptt-1 . This should behave like a mean-zero, unit variance, iid sequence. Check a time plot Turning point test Difference sign test Rank test Q-Q plot, histogram, to assess normality 12 Testing i.i.d.: Turning point test {Xt } i.i.d. implies that Xt , Xt+1 and Xt+2 are equally likely to occur in any of six possible orders: 3.5 3 2.5 2 1.5 1 0.5 0 5 10 15 20 (provided Xt , Xt+1 , Xt+2 are distinct). Four of the six are turning points. 13 Testing i.i.d.: Turning point test Define T = |{t : Xt , Xt+1 , Xt+2 is a turning point}|. ET = (n - 2)2/3. Can show T AN (2n/3, 8n/45). Reject (at 5% level) the hypothesis that the series is i.i.d. if 2n > 1.96 T- 3 8n . 45 Tests for positive/negative correlations at lag 1. 14 Testing i.i.d.: Difference-sign test S = |{i : Xi > Xi-1 }| = |{i : ( X)i > 0}|. ES = n-1 . 2 Can show S AN (n/2, n/12). Reject (at 5% level) the hypothesis that the series is i.i.d. if S- Tests for trend. (But a periodic sequence can pass this test...) n > 1.96 2 n . 12 15 Testing i.i.d.: Rank test N = |{(i, j) : Xi > Xj and i > j}|. n(n - 1) EN = . 4 Can show N AN (n2 /4, n3 /36). Reject (at 5% level) the hypothesis that the series is i.i.d. if n2 > 1.96 N- 4 Tests for linear trend. n3 . 36 16 Testing if an i.i.d. sequence is Gaussian: qq plot Plot the pairs (m1 , X(1) ), . . . , (mn , X(n) ), where mj = EZ(j) , Z(1) < < Z(n) are order statistics from N (0, 1) sample of size n, and X(1) < < X(n) are order statistics of the series X1 , . . . , Xn . Idea: If Xi N (, 2 ), then EX(j) = + mj , so (mj , X(j) ) should be linear. There are tests based on how far correlation of (mj , X(j) ) is from 1. 17 Introduction to Time Series Analysis. Lecture 13. 1. Review: Maximum likelihood estimation 2. Computational simplifications: un/conditional least squares 3. Diagnostics 4. Model selection 5. Integrated ARMA models 18 ...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online