# 18 - Introduction to Time Series Analysis Lecture 18 1...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introduction to Time Series Analysis. Lecture 18. 1. Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 1 Review: Spectral density If a time series {Xt } has autocovariance satisfying h=- |(h)| < , then we define its spectral density as f () = h=- (h)e-2ih for - < < . We have 1/2 (h) = -1/2 e2ih f () d. 2 Review: Rational spectra For a linear time series with M A() polynomial , f () = If it is an ARMA(p,q), we have 2 f () = w 2 w e 2i 2 . (e-2i ) (e-2i ) q j=1 p j=1 2 = 2 q 2 w 2 p e -2i |e-2i - zj 2 2, - pj | where z1 , . . . , zq are the zeros (roots of (z)) and p1 , . . . , pp are the poles (roots of (z)). 3 Review: Time-invariant linear filters A filter is an operator; given a time series {Xt }, it maps to a time series {Yt }. A linear filter satisfies Yt = j=- at,j Xj . time-invariant: at,t-j = j : Yt = j=- j Xt-j . causal: j < 0 implies j = 0. Yt = j=0 j Xt-j . 4 Time-invariant linear filters The operation j Xt-j j=- is called the convolution of X with . 5 Time-invariant linear filters The sequence is also called the impulse response, since the output {Yt } of the linear filter in response to a unit impulse, 1 if t = 0, Xt = 0 otherwise, is Yt = (B)Xt = j=- j Xt-j = t . 6 Introduction to Time Series Analysis. Lecture 18. 1. Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 7 Frequency response of a time-invariant linear filter Suppose that {Xt } has spectral density fx () and is stable, that is, j=- |j | < . Then Yt = (B)Xt has spectral density fy () = e 2i 2 fx (). The function (e2i ) (the polynomial (z) evaluated on the unit circle) is known as the frequency response or transfer function of the linear filter. The squared modulus, |(e2i )|2 is known as the power transfer function of the filter. 8 Frequency response of a time-invariant linear filter For stable , Yt = (B)Xt has spectral density fy () = e2i 2 fx (). We have seen that a linear process, Yt = (B)Wt , is a special case, since 2 fy () = |(e2i )|2 w = |(e2i )|2 fw (). When we pass a time series {Xt } through a linear filter, the spectral density is multiplied, frequency-by-frequency, by the squared modulus of the frequency response |(e2i )|2 . This is a version of the equality Var(aX) = a2 Var(X), but the equality is true for the component of the variance at every frequency. This is also the origin of the name `filter.' 9 Frequency response of a filter: Details Why is fy () = e2i 2 fx ()? First, y (h) = E = j Xt-j k=- j=- k Xt+h-k j j=- k=- k E [Xt+h-k Xt-j ] = j=- j k=- k x (h + j - k) = j j=- l=- h+j-l x (l). It is easy to check that |j | < and j=- h=- |x (h)| < imply that h=- |y (h)| < . Thus, the spectral density of y is defined. 10 Frequency response of a filter: Details fy () = h=- (h)e-2ih = h=- j=- j l=- h+j-l x (l)e-2ih = j=- j e2ij l=- x (l)e-2il h=- h+j-l e-2i(h+j-l) = (e2ij )fx () h=- h e-2ih = (e2ij ) fx (). 2 11 Frequency response: Examples For a linear process Yt = (B)Wt , fy () = e2i 2 2 w . For an ARMA model, (B) = (B)/(B), so {Yt } has the rational spectrum 2 fy () = w (e-2i ) (e-2i ) 2 q 2 2 = w 2 p q j=1 p j=1 |e-2i e-2i - zj 2 2, - pj | where pj and zj are the poles and zeros of the rational function z (z)/(z). 12 Frequency response: Examples Consider the moving average 1 Yt = 2k + 1 k Xt-j . j=-k This is a time invariant linear filter (but it is not causal). Its transfer function is the Dirichlet kernel 1 e-2ij (e-2i ) = Dk (2) = 2k + 1 j=-k 1 if = 0, = sin(2(k+1/2)) otherwise. (2k+1) sin() 13 k Example: Moving average Transfer function of moving average (k=5) 1 0.8 0.6 0.4 0.2 0 -0.2 -0.4 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 14 Example: Moving average Squared modulus of transfer function of moving average (k=5) 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 This is a low-pass filter: It preserves low frequencies and diminishes high frequencies. It is often used to estimate a monotonic trend component of a series. 15 Example: Differencing Consider the first difference Yt = (1 - B)Xt . This is a time invariant, causal, linear filter. Its transfer function is (e-2i ) = 1 - e-2i , so |(e-2i )|2 = 2(1 - cos(2)). 16 Example: Differencing Transfer function of first difference 4 3.5 3 2.5 2 1.5 1 0.5 0 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 This is a high-pass filter: It preserves high frequencies and diminishes low frequencies. It is often used to eliminate a trend component of a series. 17 Introduction to Time Series Analysis. Lecture 18. 1. Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 18 Estimating the Spectrum: Outline We have seen that the spectral density gives an alternative view of stationary time series. Given a realization x1 , . . . , xn of a time series, how can we estimate the spectral density? One approach: replace () in the definition f () = h=- (h)e-2ih , with the sample autocovariance (). ^ Another approach, called the periodogram: compute I(), the squared modulus of the discrete Fourier transform (at frequencies = k/n). 19 Estimating the spectrum: Outline These two approaches are identical at the Fourier frequencies = k/n. The asymptotic expectation of the periodogram I() is f (). We can derive some asymptotic properties, and hence do hypothesis testing. Unfortunately, the asymptotic variance of I() is constant. It is not a consistent estimator of f (). We can reduce the variance by smoothing the periodogram--averaging over adjacent frequencies. If we average over a narrower range as n , we can obtain a consistent estimator of the spectral density. 20 Estimating the spectrum: Sample autocovariance Idea: use the sample autocovariance (), defined by ^ 1 (h) = ^ n n-|h| t=1 (xt+|h| - x)(xt - x), for -n < h < n, as an estimate of the autocovariance (), and then use a sample version of f () = h=- (h)e-2ih , That is, for -1/2 1/2, estimate f () with n-1 ^ f () = h=-n+1 (h)e-2ih . ^ 21 Estimating the spectrum: Periodogram Another approach to estimating the spectrum is called the periodogram. It was proposed in 1897 by Arthur Schuster (at Owens College, which later became part of the University of Manchester), who used it to investigate periodicity in the occurrence of earthquakes, and in sunspot activity. Arthur Schuster, "On Lunar and Solar Periodicities of Earthquakes," Proceedings of the Royal Society of London, Vol. 61 (1897), pp. 455465. To define the periodogram, we need to introduce the discrete Fourier transform of a finite sequence x1 , . . . , xn . 22 Introduction to Time Series Analysis. Lecture 18. 1. Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 23 Discrete Fourier transform For a sequence (x1 , . . . , xn ), define the discrete Fourier transform (DFT) as (X(0 ), X(1 ), . . . , X(n-1 )), where 1 X(k ) = n n xt e-2ik t , t=1 and k = k/n (for k = 0, 1, . . . , n - 1) are called the Fourier frequencies. (Think of {k : k = 0, . . . , n - 1} as the discrete version of the frequency range [0, 1].) First, let's show that we can view the DFT as a representation of x in a different basis, the Fourier basis. 24 Discrete Fourier transform Consider the space Cn of vectors of n complex numbers, with inner product a, b = a b, where a is the complex conjugate transpose of the vector a Cn . Suppose that a set {j : j = 0, 1, . . . , n - 1} of n vectors in Cn are orthonormal: 1 if j = k, j , k = 0 otherwise. n-1 Then these {j } span the vector space Cn , and so for any vector x, we can write x in terms of this new orthonormal basis, x= j=0 j , x j . (picture) 25 Discrete Fourier transform Consider the following set of n vectors in Cn : 1 ej = e2ij , e2i2j , . . . , e2inj n n n : j = 0, . . . , n - 1 . It is easy to check that these vectors are orthonormal: ej , ek t 1 1 2it(k -j ) 2i(k-j)/n = e = e n t=1 n t=1 1 if j = k, = 1 e2i(k-j)/n 1-(e2i(k-j)/n )n otherwise n 1-e2i(k-j)/n 1 if j = k, = 0 otherwise, 26 Discrete Fourier transform where we have used the fact that Sn = n t satisfies t=1 Sn = Sn + n+1 - and so Sn = (1 - n )/(1 - ) for = 1. So we can represent the real vector x = (x1 , . . . , xn ) Cn in terms of this orthonormal basis, n-1 n-1 x= j=0 ej , x ej = j=0 X(j )ej . That is, the vector of discrete Fourier transform coefficients (X(0 ), . . . , X(n-1 )) is the representation of x in the Fourier basis. 27 Discrete Fourier transform An alternative way to represent the DFT is by separately considering the real and imaginary parts, 1 X(j ) = ej , x = n 1 = n n e-2itj xt t=1 n 1 cos(2tj )xt - i n t=1 n sin(2tj )xt t=1 = Xc (j ) - iXs (j ), where this defines the sine and cosine transforms, Xs and Xc , of x. 28 Introduction to Time Series Analysis. Lecture 18. 1. Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 29 ...
View Full Document

## This note was uploaded on 05/14/2011 for the course STAT 153 taught by Professor Staff during the Fall '08 term at Berkeley.

Ask a homework question - tutors are online