deviations Large for quadratic functionals of Gaussian processes
Wlodzimierz Bryc Department of Mathematics University of Cincinnati Cincinnati, OH 45 221 [e-mail] Amir Demboy Department of Mathematics and Department of Statistics Stanford University Stanford, CA 94 305 [e-mail] April 9, 1993 Revised September 28, 1993
Abstract
The Large Deviation Principle is derived for several unbounded additive functionals of cenR 1 tered stationary Gaussian processes. For example,Rthe rate function corresponding to T 0T Xt2dt is the Fenchel-Legendre transform of L(y) = 41 1 log(1 4 yf(s))ds, where Xt is a contin1 uous time process with the bounded spectral density f(s). The spectral density condition for quadratic functionals is weaker than known su cient conditions for bounded continuous functionals. Similar results in the discrete-time version are obtained for the energy of multivariate Gaussian processes and for the sums of p < 2 powers.
1
Introduction
Let E be a separable Banach space. Throughout most of the paper E = R, except in Proposition 2.2, where E = R2, and in Proposition 2.3, where E = Rd+1 . Suppose Sn; n > 0, are E-valued random variables. We shall say that fn 1 Sng satis es the Large Deviation Principle (LDP), if there is a lower semicontinuous rate function I : E ! 0; 1], with compact level sets I 1 ( 0; a]) for all a > 0, and such that lim inf n 1 logP(n 1Sn 2 A) n!1 for all open subsets A E; lim sup n 1 logP(n 1Sn 2 A) n!1
x2A x2A
inf I(x) inf I(x)
for all closed subsets A E. We shall work with the continuous indices n (which below are denoted by T rather than n) as well as with the discrete n = 1; 2; : : :; in Section 2.3 we shall also consider other normalizations.
Partially supported by C.P. Taft Memorial Fund y Partially supported by NSF DMS92-09712 grant and by a US-ISRAEL BSF grant. Key Words: large deviations, quadratic additive functionals, Gaussian processes. AMS (1991) Subject Classi cation: 60F10
1
For a general stationary process Xj , the LDP for the empirical measures (i.e., in the discrete time P setup corresponding to Sn = n Xj ) and the related question for bounded additive functionals j=1 P (i.e., Sn = n F(Xj ), with bounded F( )) have been studied by a number of authors under some j=1 restriction on dependence; see 10, Section 6.4] for a sample of results, and 10, Section 6.9 page 280] for relevant references. Gaussian processes were studied in 13], LDP for Gaussian elds is given in 21], see also 12] for an interesting case. Large deviations for general unbounded additive functionals of Markov chains under minimal assumptions were studied e.g. in 19]. Quadratic forms in Gaussian random variables have been studied by various asymptotic methods e.g. in statistical and electrical engineering literature; for an early paper using the saddle point method to approximate the distribution for a xed number of variables, see 18], see also 17]. There is also a number of papers on the Central Limit Theorem (CLT), see e.g. 1], 22] and the references therein. Several results directly pertinent to the LDP have appeared: 8] gives a version of the LDP restricted to certain sets and obtained using the Grenander-Szego method as employed below (and also in 2] and 6]). Their results however deal with quadratic forms in implicit way and without explicit expressions for the rate function; 2] presents the heuristic reasoning that motivated and facilitated much of this paper; in 7], the LDP given as Corollary 2.1 below is stated under an additional technical assumption; in 5] explicit rate function is found for autoregressive AR(1) processes. In this paper, the LDP is derived for several unbounded additive functionals of stationary centered Gaussian processes that posses spectral density. Of those, quadratic functionals received most attention - for electrical engineering motivation the reader is referred to 6]; motivation from control theory is presented in the introduction to 5]; statistical motivation can be read out from R 8]. 1 T X 2 dt, The following describes the contents of the paper. In Theorem 2.1 we show that T 0 t where Xt is a continuous time process with the bounded spectral density f(s),1 R satis es the LDP and the rate function is given by the Fenchel-Legendre transform of L(y) = 41 1 log(1 4 yf(s))ds. In Theorem 2.2 we show the corresponding multivariate discrete-time result. The LDP with normalization of o(n) and the quadratic rate function (corresponding to more moderate deviations) is derived in Theorem 2.3 for unbounded spectral densities (see P Section 3.7] for such results in the 10, 1 context of Cramer's theorem). In Corollary 2.2 we analyze n n jXj jp for p < 2. Proposition 2.1 j=1 points out the relevance to the CLT. In Section 2.5 we incorporate a non-zero mean in the univariate version of Theorem 2.2, thus deriving the LDP for the empirical variance. In Section 2.6 the LDP is derived for the empirical autocorrelation vector of an i.i.d. process Xj and some counter intuitive results concerning the validity of this LDP when fXj g is an AR(1) process are presented. An approach to higher order expansions is sketched in Section 2.7.
2
Results
This section contains statements of our main results. The proofs are given in Section 3, except for those results that are marked as immediate consequences of other theorems.
2.1 Continuous time
Let fXt g be a real-valued, centered, separable stationary Gaussian process with the covariance R R(t) = E(X0 Xt ) and spectral density f(s), i.e., R(t) = 1 eitsf(s)ds. 1 R T X 2 dt, M = ess sup f(s). Denote ST = 0 t Theorem 2.1 Suppose that fXtgt 0 has bounded spectral density function f(s) 2 L1(R; ds). Then 1 f T ST g satis es the LDP with the rate function I(x) = sup fxy L(y)g; (1)
1<y<1=(4 M)
2
where for y < 1=(4 M)
Z1 L(y) = 41 log(1 4 yf(s))ds: 1
(2)
As an application, suppose that Xt is the Ornstein-Uhlenbeck process, i.e., the stationary solution to a dXt = aXt + padWt, a > 0. The spectral density is f(s) = 1 a2 +s2 with M = 1=( a). Integrating 2 p2 1 expression (2) we get L(y) = 2 a 1 a 4ay, leading to I(x) = a px p1x for x > 0 and 2 4 I(x) = 1 otherwise.
2.2 Discrete time
The following result is the nite-dimensional discrete time version of Theorem 2.1.
Theorem 2.2 Let fXk gk=1;2;::: be a centered, stationary Gaussian Rd -valued sequence with the spectral density F(s) = Fi;j (s)] such that ess sup kF(s)k < 1 (where kFk denotes the operator norm associated with P matrix F, c.f. (22) below). Then for every nonnegative de nite symmetric the real matrix W, fn 1 n hXj jWXj ig satis es the LDP with the rate function j=1 I(x) = sup fxy L(y)g; (3)
1<y<1=(2M)
where M = ess sup kW1=2F(s)W1=2k and for y < 1=(2M)
1 Z 2 logdet(I 2yWF(s))ds: L(y) = 4 0
(4)
Remark 2.1 Clearly, Theorem 2.2 implies that the LDP holds also when W is a nonpositive definite symmetric real matrix. However, in Section 2.6 we give an example of W that is neither positive de nite nor negative de nite for which L(y) = 1 even when all eigenvalues of 2yWF(s)
are uniformly (in s) strictly less than 1.
The following special case of Theorem 2.2 is of interest.
where here M = ess sup f(s) and
Corollary 2.1 Let fXk gk=1;2;::: be a real-valued, centered, stationary Gaussian process with bounded 1P spectral density function f(s). Then f n n Xj2 g satis es the LDP with the rate function of (3) j=1
Z2 L(y) = 41 log(1 2yf(s))ds: 0
(5)
The next corollary follows from Corollary 2.1 if p = 2; it follows from 13] by an approximation argument if p < 2. The approximation argument does not apply to p = 2 case, because the cummulant generating function is unbounded in the latter case.
Corollary 2.2 Suppose that fXk gk=1;2;::: has continuous spectral density satisfying R02 log f(s)ds > 1 1. If p 2 then f n Pn jXj jp g satis es the LDP. j=1 Remark 2.2 Theorems 2.1 and 2.2 can be also extended to the multivariate index case (Gaussian
random elds on Rk or Z k ). Indeed, 16, Chapter 8] develops the relevant abstract results.
3
A suitably modi ed variant of the LDP holds true also when the spectral density is unbounded. P Namely, taking Sn = n Xj2 we shall show that for a certain sequence mn ! 1 random variables j=1 1 2 2 fmn ( n Sn E(X1 ))g satisfy the upper and lower bounds with exponent mn =n, i.e., 2 1 2 inf I(x) lim inf mn logP(mn ( n Sn E(X1 )) 2 A) n!1 n x2A 2 1 2 inf I(x); (6) lim sup mn logP(mn ( n Sn E(X1 )) 2 A) n
n!1 x2A
2.3 Unbounded spectral density
Theorem 2.3 Suppose that real-valued, centered stationary Gaussian process fXj gj 1 has spectral density function f(s) 2 Lq (ds), where 2 < q 1. Let fmn gbe such that n 1=q mn ! 1 (if q = 1, 1 2 assume mn ! 1), and n 1=2mn ! 0. Then fmn ( n Sn E(X1 ))g satis es the LDP (6) with the
rate function where
where A and A denote the interior and the closure of a measurable set A respectively.
2
Z =1
2 I(x) = 2x 2 ; 2
Remark 2.3 With minor changes in the statement and in the proof, Theorem 2.3 holds true both
2
0
f 2 (s)ds:
(7)
in the multivariate setup of Theorem 2.2 and in the continuous time setup of Theorem 2.1 with the same I(x), but with (7) replaced by
=
2
1
Z2
0
tr (F(s))2 ds
(8) (9)
in the former case (taking W = I) and
=4
Z1
1
f 2 (s)ds
in the latter.
2.4 Normal convergence
Lemmas 3.3 and 3.6 from the proof of the LDP yield the following CLT. At least in the univariate discrete time setup this result is known, see 1, Theorem 2], 15, Theorem 2] for a direct proof (for non-normal convergence, see 22]). Related results are given in 3, Theorem 5] and the references therein, c.f. also 20, page 58, Theorem 3].
Proposition 2.1 (i) If fXtg is a real-valued, centered, separable stationary Gaussian process with R 2 the spectral density f(s) 2 L2 (R; ds)\L1(R; ds), then p1T 0T (Xt2 E(X0 ))dt is asymptotically normal N(0; ) as T ! 1 with 2 given by (9). (ii) If fXk gk=1;2;::: is a centered, stationary Gaussian Rd -valued sequence with the spectral density F(s) = Fi;j(s)], such that tr (F(s))2 is integrable, then
1 pn
n X i=1
(hXi jXii E(hX1 jX1i))
2
is asymptotically normal N(0; ) as n ! 1 with
given by (8).
4
Many of the results presented above carry over to the case of non-centered stationary Gaussian processes by application of the contraction principle. For concreteness, consider the setup of Corollary 2.1, i.e. let fXj g be a real-valued centered stationary Gaussian process. Proposition 2.2 Suppose that spectral density f( ) is di erentiable. Let Sn = Pn Xj ; Pn Xj2 ]0. j=1 j=1 Then fn 1Sn g satis es the LDP (in R2 ) with the rate function x2 1 J(x1 ; x2) = I(x2 x2) + 2f(0) ; (10) 1 where I( ) is the rate function given by (3) and (5), and if f(0) = 0 then J(x1; x2) = 1 for x1 6= 0 while J(0; x2) = I(x2 ). Applying the contraction principle (see 10, Theorem 4.2.1]) with respect to the continuous function g(x1 ; x2) = x2 Pn 1 + 2 : R2 ! R, we see that for a non-centered process Yj = Xj + , the + 2x sequence fn 1 j=1 Yj2g satis es the LDP (in R) with rate function
2y ~ J(z) = f(x ;x ):z=g(x ;x )g J(x1; x2) = sup fzy 1 2yf(0) L(y)g ; inf 12 12 y<1=(2M)
2.5 Non-centered processes and the LDP for the empirical variance
where M = ess sup f(s) and L(y) given by (5), compare also 2, page 361]. Similarly, applying the 2 contraction principle with respect to the continuous function h(x1 ; x2) = x2 x1 results with the n satisfying the LDP with the rate function I( ) given by (3) and (5) empirical variance of fXj gj=1 P (i.e. the same rate as for fn 1 n Xj2 g). j=1
(j) (j) nj For j 0, let Sn = k=1 Xk Xk+j . Then n 1 Sn is the j-th empirical autocorrelation based on (0) (d) the sample of size n. For xed d 1 let Sn = Sn ; : : :; Sn ] 2 Rd+1 . If f( ) is the spectral density of fXj g, denote f(s) = f(s); f(s) cos s; : : :; f(s) cos sd]0 2 Rd+1: 1 Proposition 2.3 Suppose that fXk gk=1;2;::: are i.i.d. N(0,1) random variables. Then f n Sng sat-
2.6 The empirical autocorrelation vector
P
is es the LDP with the rate function
I(x) = supfhxjyi L(y) : y 2 Dg;
where and for y 2 D
D = fy 2 Rd+1 : sup hyjf(s)i < 1=2g;
0
However, the example below shows that for d = 1 and for every AR(1) process with 0 < jaj < 1, (11) 1 is false for some y 2 D. Hence, in these cases even if f n Sng satis es the LDP, the rate function cannot be given by the expression as in Proposition 2.3.
Remark 2.4 The proof of Proposition 2.3 (with the same formula for the rate function) extends to any di erentiable spectral density f(s) provided that for all y 2 D lim sup n 1 logE(exp(hyjSni)) < 1 : (11) n!1
Z2 log(1 2hyjf(s)i)ds : L(y) = 41 0
s
2
5
Example 2.1 Let Xk be an AR(1) process (with 0 = 1, 1 = 0 and 0 < jaj < 1) corresponding to ri = E X0 Xi ] = ai =(1 a2 ) for i = 0; 1; : : : and f(s) = 1=(1 + a2 2a cos s). Therefore y = 1+a2; 2a]0 2 D for every < 1=2. Let Rn denote the covariance matrix of X = X1; : : :; Xn]0 and let Yn be the n n symmetric Toeplitz matrix corresponding to y0 = (1 + a2), y1 = a and yi = 0 for all 1 < i n 1. Since Rn 1 r0; : : :; rn 1]0 = 1; 0; : ::; 0]0, we have for > (1 a2)=2 and all n large enough
h r0; : : :; rn 1]j(Rn
1
2Yn ) r0; : : :; rn 1]0i = r0
2 (1 + a )
2
n1 X i=0
ri + 4 a
2
n2 X i=0
riri+1 < 0 ;
(1) (0) implying that E(exp( (1 + a2 )Sn 2 aSn )) = 1 (see Lemma 3.1). Note that the above expression is related to Theorem 2.2. Indeed, (0) (1 + a2 )(Sn 2 Xn (1 2 (1) )X1 ) 2 aSn =
n1 X j=1
hXj jW Xj i
where Xj = Xj ; Xj+1]0 2 R2 and
a )(1 + a2 ) : Considering 0, W is nonnegative de nite i 2 a2=(1 + a2 ); 1=(1 + a2 )]. For this range of it follows by applying Lemma 3.6 to Yj = W1=2Xj that for all < 1=2, (0) 2 2 (1) lim n 1 log E(exp( (1 + a2)(Sn Xn (1 )X1 ) 2 aSn )) = 1 log(1 2 ) : (12) n!1 2 It can also be veri ed that for every > 1=(1+a2) the left side of (12) is in nite for some 2 (0; 1=2), while the eigenvalues of W F(s) (which are 0 and ) are independent of .
W=
(1 + a2 ) a (1
Remark 2.5 The example shows that the large deviations of the empirical autocorrelation vector are sensitive to boundary e ects (the choice of ), and that Theorem 2.2 does not extend to matrices W which are neither nonnegative de nite nor nonpositive de nite.
The following result comes essentially form 16, page 76]. Together with saddle point approximation, it can be used to nd higher order asymptotic expansions for probabilities of "regular enough" sets in Corollary 2.1. We do not pursue this possibility here.
2 spectral density f(s) and M = ess sup f(s). Let Sn = k=1 Xk and L(y) be de ned by (5). Then for all y < 1=(2M) the sequence fexp( nL(y))E(exp(ySn ))g is monotonically nonincreasing. If in addition f(s) is di erentiable and for some > 0 the function f 0 (s) is uniformly Lipschitz continuous with exponent then
2.7 Exact asymptotic
Corollary 2.3 Suppose fXk gk 1 is a centered, real-valued stationary Gaussian sequence with bounded Pn
ZZ lim exp( nL(y))E(exp(ySn )) = exp(L(y) 21 jh0y (z)j2d ); n!1 jzj 1
Z2 ze is hy (z) = 41 log(1 2yf(s)) 1 + ze is ds ; 1 0 and (dz) is the surface measure on the unit disc in C.
where
6
3
Proofs
We shall need the following well known elementary result. Lemma 3.1 Suppose X = X1 ; : : :; Xn]0 is a real valued centered Gaussian vector with the covariance matrix R and let M be a symmetric real valued n n-matrix. Then with 1 ; : : :; n the
eigenvalues of the matrix MR
n 1X log E exp(zhXjMXi) = 2 log(1 2z j ) j=1
for z 2 C such that maxj fRe(z) j g < 1=2. Furthermore, logE exp(yhXjMXi) = 1 for y 2 R such that maxj fy j g 1=2.
With X = R1=2Z and Z a standard multivariate normal, Lemma 3.1 follows by direct integration of the density of Z.
Lemma 3.2 If fYj g are i.i.d. r.v. with mean zero, nite second moment and positive probability density function at 0, then for each > 0 there is > 0 such that
inffP(j
Proof : Denote
2
1 X
i=1
ki Yi j < ) :
1 X
i=1
jkij 1g
:
= E(Y 2 ) and x the sequence fkig. Without loss of generality, we may assume P that jkij jki+1j for all i 1. Note that then the condition j jkj j 1 implies that jkj j 1=j for all j 1. Consequently, for every r 1 by Chebyshev's inequality we have P(j
1 X
i=r+1
ki Yij < ) 1
2 2
j=r+1
1 X1 j2 :
(13)
Note that one can nd r0 = r0 ( ) such that the right hand side of (13) is strictly positive. Choose now such r0( =2). By independence we have P(j
1 X
i=1
1 X
i=1
kiYi j < ) P(j
r0 X i=1
ki Yij < =2)P(j
1 X
i=r0 +1
ki Yi j < =2)
and, since jkij 1, using (13) we get P(j ki Y i j < ) P(1max jYij < =(2r0 ))P(j ir
0
1 X
i=r0 +1
0 1 1 4 2 X 1 A =: : P(jY1j < =(2r0 ))r0 @1 2 2 j=r0 +1 j
kiYi j < =2)
This ends the proof with > 0 as de ned above. For complex z with Re(z) < 4 1M , let LT (z) = log E(exp(zST )). The following Lemma was motivated by a heuristic argument in 2]. 7
3.1 Proof of Theorem 2.1
Z1 1 log(1 4 zf(s))ds: lim T LT (z) = 41 T!1 1 Proof : For T > 0, denote by j = j (T) the eigenvalues of ZT
0
Lemma 3.3 Under the assumptions of Theorem 2.1, for Re(z) < 4 1M we have
R(t s)g(s)ds = g(t) 2 L2 ( 0; T ])
(14)
and let ej = ej (t) 2 L2 ( 0; T ]; dt) be the corresponding orthonormal eigenfunctions. Since by P Mercer's theorem, R(t s) = j j ej (t)ej (s) with positive and summable eigenvalues f j g, we Pp have the Karhunen-Loeve expansion Xt = j j j ej (t), where j are i.i.d. N(0,1). Note that sup j = j
g2L2 ;kgk=1
sup
ZT
0
g(t)dt
ZT
0
g(u)du
Z1
1
ei(t u)sf(s)ds:
Since for T < 1 each square-integrable g( ) is integrable, we may switch the order of integration, which gives Z1 ZT sup j M j g(t)eits dtj2ds = 2 M; (15) where the last equality is by Plancherel's theorem. Therefore Re(z) < 1=(4 M) 1=(2 j ) and 1 1 L (z) = 1=T log E exp(zS )] = 1=(2T) X log(1 2z ) = 1 Z 2 M log(1 2zx) (dx); (16) j T T TT 20 j=1 where T (dx) := 1=T j j (dx) denotes the distribution of the eigenvalues on 0; 2 M]. Fix z and choose > 0 such that 2jzj < 1 and such that fs : 2 f(s) = g is of Lebesgue measure zero. By 16, page 139] for k = 1; 2; : : : we have lim T!1
j
1
0
P
Z2M
0
xk T (dx) = (2 )k
1
Z1
and also for every bounded continuous F( )
1
f k (s)ds;
(17)
1Z F(2 f(s))ds: (18) lim F(x) T (dx) = 2 T!1 fs:2 f(s) g Let Pk (x) be the k-th Taylor polynomial for x 7! log(1 2zx). Notice that from (17) and (18), for each xed k we get Z Z Pk (x) T (dx) ! 21 Pk (2 f(s))ds: (19)
Z2M
Clearly, for 0 x
we have
0
fs:2 f(s) g
jPk (x) log(1 2zx)j = j
Given > 0 choose k > 2jzj(1 2jzj ) T > T0 we have
1
1 X
1
j=k+1
k+1 1 (2zx)j =jj < k (2xjzj) 1 2jzj
1 2xjzj k 1 2jzj :
1
. Then by (19) choose T0 = T0 (k) such that for all
:= j
Z
0
Z Pk (x) T (dx) 21 Pk (2 f(s))dsj < fs:2 f(s) g
8
and by (17) (with k=1)
x T (dx) < 2R(0): Enlarging T0 if necessary, by (18) we may also ensure Z2M 1Z log(1 4 zf(s))dsj < log(1 2zx) T (dx) 2 2 := j fs:2 f(s) g for all T > T0. Therefore for all T > T0 we have Z2M 1 Z 1 log(1 4 zf(s))dsj j log(1 2zx) T (dx) 2 0 1
0 1+ 2+
Z2M
Z2M
0
x T (dx) +
Z1
Remark 3.1 By the induced convergence for analytic functions, from Lemma 3.3 it follows that
1
f(s)ds < (2 + 3R(0)) :
Z 1 f(s) d d ds T 1 dy LT (y) ! dy L(y) = 1 1 4 yf(s)
in nite for y > 1=(4 M), and by Lemma 3.3 L(y) exists and given by (2) for all y < 1=(4 M). De ne L(1=(4 M)) = limy%1=(4 M) L(y) (which by monotone convergence coincides with L(1=(4 M)) of (2)), and note that by the monotonicity of LT (y) with respect to y lim inf T 1 LT (yT ) L(1=(4 M)) : (20) T!1; y !1=(4 M)
T
for all y < 4 1M (this can be also veri ed directly using 16, page 139]). Remark 3.2 Let 1(T) be the maximal eigenvalue of (14). Then 1 (T) 2 M by (15), and therefore by 16, page 139] one has 1 (T) ! 2 M as T ! 1. Proof of Theorem 2.1: By Remark 3.2 and Lemma 3.1 it follows that L(y) = limT!1 T 1LT (y) is
If L(1=(4 M)) = 1, then the result follows by the Gartner-Ellis Theorem (see 10, Theorem d 2.3.6]), for then (20) holds with equality, and L( ) is steep, i.e., limy%1=(4 M) dy L(y) = 1. Since in general 1 L (1=(4 M)) converges), we follow instead the this is not the case (and it is not even clear that T T strategy of parameter dependent change of measure, as outlined in 11]. Indeed, by the monotonicity of LT ( ) it follows that 11, (2.13) and (2.15)] hold. Excluding the trivial case of zero spectral density, since L0 (y) > 0 is non-decreasing, there is c > 0 such that L0 (y) ! c as y % 1=(4 M), and examining 11, Proposition 2.14] we see that the LDP with the rate function of (1) holds even for L(1=(4 M)) < 1 as soon as L( ) is steep, i.e. c = 1. Turning to deal with L( ) which is not steep, i.e. c < 1, observe that then I( ) of (1) is continuous at x = c and it is simple to check that for xc I(x) = 4 xM L( 4 1M ): Thus, by 11, Proposition 2.14], su ces to show that for all x > c and all > 0 small enough 1 (x + ) + L( 1 ); 1 (21) lim inf T log P(jT 1ST xj < ) T!1 4M 4M in order to complete the proof of the theorem. To this end, let 1 (T) 2 (T) : : : n (T) : : : be the eigenvalues of (14) and for y < 1=(2 1) let kj (y; T ) = T(1 j2y ) : j 9
d Since T 1 dy LT (y) = j kj (y; T ) is monotone in y and approaches 1 as y approaches 1=(2 1), P there exists yT < 1=(2 1 (T)) such that 1 kj = x for kj = kj (yT ; T ). Moreover, for each xed j=1 d d y < 1=(4 M), by Remark 3.1 limT T 1 dy LT (y) = dy L(y) c < x, while lim supT yT 1=(4 M) by Remark 3.2; hence yT ! 1=(4 M). For yT as above, de ne the measure QT via dQT = exp(y S L (y )); TT TT dP and let VT denote the r.v. (T 1 ST x) under measure QT . Note that by (16) the Laplace transform of VT is given by 1 Y p E esVT ] = exp( ski )= 1 ski ;
P
where ki = ki(yT ; T ). Therefore VT has the representation VT =
1 X
j=1
i=1
kj (Zj2 1)
with Zj i.i.d. normal N(0,1), and by Lemma 3.2 we deduce that QT (jT 1ST xj < ) for all > 0 and some = ( ) > 0 which is independent of T. Since yT 0 for all large T, Z dP T 1 log P(jT 1ST xj < ) = T 1 log( dQ 1jT 1 ST xj< dQT ) T T 1 log QT (jT 1ST xj < ) yT (x + ) + T 1LT (yT ); and the lower bound (21) follows from (20). Throughout this proof we consider Rn, n 1 as Hilbert subspaces of `2 with the inherited norms. For an n n-matrix A, we consider the usual operator norm
3.2 Proof of Theorem 2.2
and the Hilbert-Schmidt norm jAj = tr (AA0 ) (with the usual convention that A0 is the conjugate transpose of the matrix A). It is well known that jABCj kAk jBj kCk, and that kAk jAj, see e.g. 14, Section XI.6]. The distribution of the eigenvalues f 1; : : :; ng of A is the discrete probability measure n X n (dx) = n 1 j (dx) (either on R or on C, depending on whether A is symmetric, or not). The following result is known. Lemma 3.4 ( 16, p 105]) Suppose the n n matrices An and Bn have the distribution of the
eigenvalues n and n respectively and assume that
n j=1
p
kAk = sup kAyk ; y2Rn n0 kyk
(22)
sup(kAnk + kBnk) < 1; lim n 1jAn Bn j2 = 0: R R xk (dx)j = 0 for every k = 1; 2; : : : Then limn!1 j xk n(dx) n
n!1
(23) (24)
and
10
Let Rn = cov(X0 ; Xn) be the d d-covariance matrices, and let n be the distribution of the eigenvalues of the block-Toeplitz nd nd matrix 2R R1 : : : Rn 1 3 0 6 R1 R0 : : : Rn 2 7 7 (25) An = 6 .. 6. . . . ... 7 : 5 4 R (n 1) R (n 2) : : : R0 The asymptotic of n follows by extending the argument of 16, page 113] as follows. Lemma 3.5 If M = ess supkF(s)k < 1 then supn kAnk M. Moreover, for any a < b such that m(s : j (s) = a) = m(s : j (s) = b) = 0 for j = 1; : : :; d, lim ( a; b]) = (2 d) n!1 n
where m is Lebesgue measure on 0; 2 ] and
1 1
d X j=1
m(s : a < j (s) < b) ;
2
(26) 0 are the eigenvalues of
usual argument for circulant matrices shows that for j = 1; : : :; d; k = 0; : : :; n 1 the nd-dimensional vectors (vj;k ; e2 ik=nvj;k ; : : :; e2 ik(n 1)=nvj;k ) are the linearly independent eigenvectors of Cn;A corresponding to the eigenvalues j;k ; therefore those are all the eigenvalues of Cn;A . Consequently, kCn;A k sups kFA (s)k and since Z 2 sin2(A(s t)=2) FA (s) = 21 F(t)dt ; A sin2 ((s t)=2) 0 clearly, 1 Z 2 sin2(At=2) dt = M : (27) sup kFA (s)k sup kF(s)k 2 A sin2 (t=2) s s 0 We turn now to prove that kAnk M and kBn;A k M. To this end, x n, pick xj 2 Rd and write x = (xj ) as a column vector. Then,
F(s) (recall that F(s); 0 s 2 ; are Hermitian, nonnegative de nite matrices). b b Proof : For (n 1)=2 A 1 let Rk = (1 k=A)Rk for k = 0; : : :; A and Rk = 0 for k > A, with b k = R0k. Let Bn;A be the block-Toeplitz nd nd matrix constructed as in (25) but with the b R b blocks Rk instead of Rk . Let Cn;A be the block-circulant matrix associated with Bn;A, i.e. using the P b b blocks Rk modn in (25) instead of Rk . Let FA (s) = A A e iksRk , with f j;k gj=1;:::;d denoting k= d the corresponding eigenvectors. The the eigenvalues of FA (2 k=n), k = 0; : : :; n 1 and vj;k 2 R
(s)
(s)
d (s)
hxjAn xi = (2 )
1
Z2 X n
0
Z 2 X m=1 n k e iksxk k2 ds) = Mkxk2 : sup kF(s)k( 21 0s2
k=1
0
h
X e iksxk jF(s) eims xm ids (2 )
n k=1
1
Z2 X n
0
k
k=1
e iksxk k2 kF(s)kds
By a similar argument we have for n > A
hxjBn;A xi = (2 )
1
Z2 X n
0
h
This shows that matrices An and Bn;A and Cn;A satisfy (23) for every choice of A (n 1)=2. 11
X e iksxk jFA(s) eims xm ids kxk2 sup kFA (s)k Mkxk2 : s m=1 k=1
n
By applying Parseval's relation elementwise one has
1 X
j= 1
jRj j = (2 )
2
1
Z2
0
jF(s)j2ds d M 2 :
1 X
j=A+1
Since for every n > A we have n 1 jAn Bn;Aj2 2
A X j=1
(j=A)2 jRj j2 + 2
jRj j2 ;
by Kronecker's Lemma it follows that n 1 jAn Bn;Aj2 can be made arbitrarily small (uniformly in n > A) by choosing A large enough. Therefore, by choosing rst A large and then n large enough, we can make sure that (24) holds both for jAn Bn;A j and for jBn;A Cn;A j since
jBn;A Cn;A j2 2A
A X j=1
jRj j2 Ad M 2 :
Consequently, by Lemma 3.4 the asymptotic of n is the same as the asymptotic of the distribution of the eigenvalues of Cn;A provided we let n ! 1 rst and then take A ! 1. Fix a positive integer `. In view of the continuity of FA (s) we have for any xed A 1 lim n n!1 Also
1
n X
k=1
Z tr (FA (2 k=n)` ) = 21
d(2 ) d` M
2 1
2
0
tr (FA (s)` )ds :
j(2 )
and since,
1
Z2
0
tr
(FA (s)`
F(s)` )dsj2
Z2
0
jFA (s)` F(s)` j2ds
1
2(` 1)
(2 )
Z2
0
jFA (s) F(s)j2 ds ; jRj j2 ;
(2 )
1
Z2
0
jFA (s) F(s)j2 ds = 2
A X j=1
(j=A)2 jRj j2 + 2
1 X
j=A+1
R we have for A ! 1 that 02 tr (FA (s)` F(s)` )ds ! 0, leading to n X 1 Z2 1 `
A!1 n!1
lim lim n
k=1
tr (FA (2 k=n) ) = 2
0
tr (F(s)` )ds :
With the above holding for every positive integer `, the limit (26) follows by 16, page 105]. P Let Sn = n hXj jXj i and for complex z, let Ln (z) = log E(exp(zSn )). j=1 1 Lemma 3.6 If sups kF(s)k = M < 1, then the limit limn!1 n Ln(z) exists for every z in the 1 and half-plane Re z < 2M 1 L (z) = 1 Z 2 logdet(I 2zF(s))ds : lim n!1 n n 40 12 (28)
Remark 3.3 For d = 1 this lemma is known, see 6, page 105], or 7, Example 3.1 a)].
Proof : Clearly,
Sn = X1 ; : : :; Xn ] X1; : : :; Xn]0: Therefore by Lemma 3.1, for Re(z) < 1=(2 maxj j ) n Ln (z) = 1=(2n)
1
nd X j=1
log(1 2z j );
where f j g are the eigenvalues of the symmetric nonnegative de nite matrix An. Lemma 3.5 implies that maxj j = kAnk M for all n, and by (26) actually kAnk ! M as n ! 1. Consequently, (28) follows by applying (26) and observing that
ZM n 1 Ln(z) = d 2 0 log(1 2zx) n (dx) :
y < 1=(2M)
Remark 3.4 By the induced convergence for analytic functions, from Lemma 3.6 it follows that for
d d L (y) ! d L(y) = 1 X Z 2 j (s) n dy n dy 2 j=1 0 1 2y j (s) ds;
1
where j (s); j = 1; : : :; d are the (nonnegative) eigenvalues of F(s). (This claim can also be veri ed directly from (26).) Proof of Theorem 2.2: For W an identity matrix, the proof repeats the reasoning from the proof of Theorem 2.1. Indeed, by Lemma 3.6, n 1 Ln (y) converges to L(y) of (4) for y < 1=(2M), while by Lemmas 3.1 and 3.5, for y > 1=(2M)
L(y) = n!1 n 1 Ln (y) = 1 : lim Excluding the trivial case of zero spectral density, notice that L0 (y) > 0 is monotonically increasing for y < 1=(2M), and let c > 0 be such that L0 (y) ! c as y % 1=(2M). De ne L(1=(2M)) = limy%1=(2M) L(y). Since 11, (2.13) and (2.15)] hold by the monotonicity of Ln ( ), if L(y) is steep, i.e. c = 1, then the LDP with the rate function I( ) of (3) and (4) follows by 11, Proposition 2.14] (even if n 1Ln (1=(2M)) fails to converge). If L( ) is not steep then I(x) is continuous at x = c and 1 x I(x) = 2M L( 2M ) for all x c. Letting f j g denote the nonnegative eigenvalues of the matrix An, the n-dependent change ofP measure via dQn = exp(yn Sn Ln (yn )) results with n 1Sn x (under Qn ) dP having the representation nd kj (Zj2 1) with Zj i.i.d. normal N(0; 1) and kj = j =(n(1 2yn j )), j=1 P where yn < 1=(2 maxj j ) chosen such that nd kj = x. Since maxj f j g = kAn k ! M as n ! 1 j=1 it follows by Remark 3.4, that limn yn = 1=(2M) and the proof of the large deviations lower bound for x > c is completed by applying Lemma 3.2 (note that lim infn n 1Ln (yn ) L(1=(2M))). For any W nonnegative de nite symmetric real matrix, we have W = W1=2W1=2 with W1=2 also nonnegative symmetric real matrix. Hence hXj jWXj i = hYj jYj i for j = 1; 2; : : :, where Yj = W1=2Xj is a stationary process of bounded spectral density W1=2F(s)W1=2 . Therefore, the general case follows by applying the above proof to the process fYj g.
Remark 3.5 For d = 1, by Lemma 3.1 and 16, pages 38, 44], n 1 logE(exp((2M) 1 Pn Xj2 )) j=1 converges as n ! 1 to L(1=(2M)) of (5). The validity of this result in the general context of
Theorem 2.2 is not addressed here.
13
The proof is based on the Gartner-Ellis Theorem (c.f. 10, Theorem 2.3.6 and Remark (a)]) used with the normalization an = m2 =n ! 0. n We shall need the following estimate for the maximal eigenvalue of the covariance matrices. Lemma 3.7 If 1 q 1 then there is1 C < 1 such that for all n > 1 if An is the covariance matrix of X1 ; : : :; Xn ]0 then kAn k Cn q : Proof : Let x = x1; : : :; xn]0 be P that kxk = 1 and kAn k = hxjAnxi: Then, denoting 1=p+1=q = such R RP P 1; we have kAn k = 21 02 f(s)j xj eijsj2 ds kfkq ( 21 02 j xj eijsj2pds)1=p C( jxj j)(2p 2)=p Cn1=q : 1 2 Proof of Theorem 2.3: Denote Tn = mn ( n Sn EX1 ) and as previously, let j = j (n); 1 j n; be the eigenvalues of the covariance of X1 ; : : :; Xn . Since by Lemma 3.7 and the choice of mn maxj j =mn ! 0, for every y 2 R and for all n n0 (y) we have
n X 2 log E exp(nmn 2 yTn ) = ynmn 1 EX1 1 log(1 2y j =mn) : 2 j=1
3.3 Proof of Theorem 2.3
Notice that by Taylor's Theorem for jwj < 1 log(1 w) = w (1=2)w2(1 tw) 2; where t = t(w) 2 0; 1]. This is applied here to wj = 2y j =mn which by Lemma 3.7 satis es supj jwj j ! 0 as n ! 1, and hence, j1 t(wj )wj j ! 1 uniformly in 1 j n. This shows that the limit of m2 n 1 log E exp(nmn 2 yTn ) n is the same as that of n n X2 X 1 2 2 1 ymn (n j E(X1 )) + y n j
P Clearly, n
n
j=1 j
1
2 = tr An = nE(X1 ), and 1 2 j = n tr An = 2
j=1
j=1
n X j=1
n1 X k=
n 2 2 Notice that by Parseval's identity k= 1 (n 1) rk ! 1 1 rk = 2 =2 as n ! 1. On the other k= Pn 1(k=n)r2 ! 0 as n ! 1 leading to hand, by Kronecker's Lemma k=1 k 1 lim m2 n 1 log E exp(nmn 2 yTn ) = 2 y2 2: n!1 n This ends the proof by the Gartner-Ellis Theorem.
P
(n 1)
2 (1 jkj=n)rk =
n1 X k=
(n 1)
2 rk
2
n1 X k=1
2 (k=n)rk
P
For f(s) or kF(s)k bounded, the CLT follows immediately from Lemmas 3.3 and 3.6 by a simple complex analysis argument given in 4, Proposition 1]. In general, for every M < 1, we let Xt = Yt + Zt in the continuous time setup and Xk = Yk + Zk in the discrete time setup; in the former case Yt and Zt are independent, real-valued, centered, separable stationary Gaussian processes with spectral densities fy (s) = min(f(s); M) and fz (s) = f(s) fy (s), while in the latter Yk and Zk 14
3.4 Pro...