Unformatted text preview: Introduction to Time Series Analysis. Lecture 19.
1. Review: Spectral density estimation, sample autocovariance. 2. The periodogram and sample autocovariance. 3. Asymptotics of the periodogram. 1 Estimating the Spectrum: Outline We have seen that the spectral density gives an alternative view of stationary time series. Given a realization x1 , . . . , xn of a time series, how can we estimate the spectral density? One approach: replace () in the definition f () =
h= (h)e2ih , with the sample autocovariance (). ^ Another approach, called the periodogram: compute I(), the squared modulus of the discrete Fourier transform (at frequencies = k/n).
2 Estimating the spectrum: Outline These two approaches are identical at the Fourier frequencies = k/n. The asymptotic expectation of the periodogram I() is f (). We can derive some asymptotic properties, and hence do hypothesis testing. Unfortunately, the asymptotic variance of I() is constant. It is not a consistent estimator of f (). 3 Review: Spectral density estimation
If a time series {Xt } has autocovariance satisfying h= (h) < , then we define its spectral density as f () =
h= (h)e2ih for  < < . 4 Review: Sample autocovariance
Idea: use the sample autocovariance (), defined by ^ 1 (h) = ^ n
nh t=1 (xt+h  x)(xt  x), for n < h < n, as an estimate of the autocovariance (), and then use
n1 ^ f () =
h=n+1 (h)e2ih ^ for 1/2 1/2. 5 Discrete Fourier transform
For a sequence (x1 , . . . , xn ), define the discrete Fourier transform (DFT) as (X(0 ), X(1 ), . . . , X(n1 )), where 1 X(k ) = n
n xt e2ik t ,
t=1 and k = k/n (for k = 0, 1, . . . , n  1) are called the Fourier frequencies. (Think of {k : k = 0, . . . , n  1} as the discrete version of the frequency range [0, 1].) First, let's show that we can view the DFT as a representation of x in a different basis, the Fourier basis. 6 Discrete Fourier transform
Consider the space Cn of vectors of n complex numbers, with inner product a, b = a b, where a is the complex conjugate transpose of the vector a Cn . Suppose that a set {j : j = 0, 1, . . . , n  1} of n vectors in Cn are orthonormal: 1 if j = k, j , k = 0 otherwise.
n1 Then these {j } span the vector space Cn , and so for any vector x, we can write x in terms of this new orthonormal basis, x=
j=0 j , x j . (picture) 7 Discrete Fourier transform
Consider the following set of n vectors in Cn : 1 ej = e2ij , e2i2j , . . . , e2inj n
n n : j = 0, . . . , n  1 . It is easy to check that these vectors are orthonormal: ej , ek
t 1 1 2it(k j ) 2i(kj)/n = e = e n t=1 n t=1 1 if j = k, = 1 e2i(kj)/n 1(e2i(kj)/n )n otherwise n 1e2i(kj)/n 1 if j = k, = 0 otherwise, 8 Discrete Fourier transform
where we have used the fact that Sn = n t satisfies t=1 Sn = Sn + n+1  and so Sn = (1  n )/(1  ) for = 1. So we can represent the real vector x = (x1 , . . . , xn ) Cn in terms of this orthonormal basis,
n1 n1 x=
j=0 ej , x ej =
j=0 X(j )ej . That is, the vector of discrete Fourier transform coefficients (X(0 ), . . . , X(n1 )) is the representation of x in the Fourier basis. 9 Discrete Fourier transform
An alternative way to represent the DFT is by separately considering the real and imaginary parts, 1 X(j ) = ej , x = n 1 = n
n e2itj xt
t=1 n 1 cos(2tj )xt  i n t=1 n sin(2tj )xt
t=1 = Xc (j )  iXs (j ), where this defines the sine and cosine transforms, Xs and Xc , of x. 10 Periodogram
The periodogram is defined as I(j ) = X(j ) = 1 n
n 2 2 e2itj xt
t=1 2 2 = Xc (j ) + Xs (j ). 1 Xc (j ) = n 1 Xs (j ) = n n cos(2tj )xt ,
t=1 n sin(2tj )xt .
t=1 11 Periodogram
Since I(j ) = X(j )2 for one of the Fourier frequencies j = j/n (for j = 0, 1, . . . , n  1), the orthonormality of the ej implies that we can write n1 j=0 n1 j=0 x x = = n1 j=0 X(j )ej n1 j=0 X(j )ej X(j )2 = I(j ). For x = 0, we can write this as 1 2 x = ^ n 1 2 xt = n t=1
n n1 I(j ).
j=0 12 Periodogram
This is the discrete analog of the identity
1/2 2 x = x (0) =
1/2 fx () d. (Think of I(j ) as the discrete version of f () at the frequency j = j/n, and think of (1/n) j as the discrete version of d.) 13 Estimating the spectrum: Periodogram
Why is the periodogram at a Fourier frequency (that is, = j ) the same as computing f () from the sample autocovariance? Almost the samethey are not the same at 0 = 0 when x = 0. But if either x = 0, or we consider a Fourier frequency j with j {1, . . . , n  1}, . . . 14 Estimating the spectrum: Periodogram
1 I(j ) = n 1 = n 1 = n
n 2 e
t=1 n t=1 2itj xt 1 = n n t=1 2 e2itj (xt  x) n e2itj (xt  x) t=1 e2itj (xt  x) n1 s,t e2i(st)j (xs  x)(xt  x) = (h)e2ihj , ^
h=n+1 where the fact that j = 0 implies n e2itj = 0 (we showed this t=1 when we were verifying the orthonormality of the Fourier basis) has allowed us to subtract the sample mean in that case. 15 Asymptotic properties of the periodogram
We want to understand the asymptotic behavior of the periodogram I() at a particular frequency , as n increases. We'll see that its expectation converges to f (). We'll start with a simple example: Suppose that X1 , . . . , Xn are i.i.d. N (0, 2 ) (Gaussian white noise). From the definitions, 1 Xc (j ) = n
n cos(2tj )xt ,
t=1 1 Xs (j ) = n n sin(2tj )xt ,
t=1 we have that Xc (j ) and Xs (j ) are normal, with EXc (j ) = EXs (j ) = 0. 16 Asymptotic properties of the periodogram
Also, 2 Var(Xc (j )) = n
n cos2 (2tj ) 2 . (cos(4tj ) + 1) = 2 t=1 = 2n Similarly, Var(Xs (j )) = 2 /2. t=1 2 n 17 Asymptotic properties of the periodogram
Also, 2 Cov(Xc (j ), Xs (j )) = n
n cos(2tj ) sin(2tj ) sin(4tj ) = 0,
t=1 = 2n Cov(Xc (j ), Xc (k )) = 0 Cov(Xs (j ), Xs (k )) = 0 Cov(Xc (j ), Xs (k )) = 0. for any j = k. t=1 2 n 18 Asymptotic properties of the periodogram
That is, if X1 , . . . , Xn are i.i.d. N (0, 2 ) (Gaussian white noise; f () = 2 ), then the Xc (j ) and Xs (j ) are all i.i.d. N (0, 2 /2). Thus, 2 2 2 2 I(j ) = 2 Xc (j ) + Xs (j ) 2 . 2 2 So for the case of Gaussian white noise, the periodogram has a chisquared distribution that depends on the variance 2 (which, in this case, is the spectral density). 19 Asymptotic properties of the periodogram
Under more general conditions (e.g., normal {Xt }, or linear process {Xt } with rapidly decaying ACF), the Xc (j ), Xs (j ) are all asymptotically independent and N (0, f (j )/2). Consider a frequency . For a given value of n, let (n) be the closest ^ Fourier frequency (that is, (n) = j/n for a value of j that minimizes ^   j/n). As n increases, (n) , and (under the same conditions that ^ ensure the asymptotic normality and independence of the sine/cosine transforms), f (^(n) ) f (). (picture) In that case, we have 2 2 d (n) 2 2 I(^ ) = Xc (^(n) ) + Xs (^(n) ) 2 . 2 f () f () 20 Asymptotic properties of the periodogram
Thus, EI(^(n) ) = 2 f () 2 2 E Xc (^(n) ) + Xs (^(n) ) 2 f () f () 2 2 E(Z1 + Z2 ) = f (), 2 where Z1 , Z2 are independent N (0, 1). Thus, the periodogram is asymptotically unbiased. 21 Introduction to Time Series Analysis. Lecture 19.
1. Review: Spectral density estimation, sample autocovariance. 2. The periodogram and sample autocovariance. 3. Asymptotics of the periodogram. 22 ...
View
Full Document
 Fall '08
 Staff
 Fourier Series, Covariance, Variance, DFT, CN

Click to edit the document details