time series.pdf - Springer Texts in Statistics RobertH.Shumway David S Stoffer Time Series Analysis and Its Applications With R Examples Fourth Edition

time series.pdf - Springer Texts in Statistics...

This preview shows page 1 out of 567 pages.

You've reached the end of your free preview.

Want to read all 567 pages?

Unformatted text preview: Springer Texts in Statistics Robert H. Shumway David S. Stoffer Time Series Analysis and Its Applications With R Examples Fourth Edition Springer Texts in Statistics Series Editors Richard DeVeaux Stephen E. Fienberg Ingram Olkin More information about this series at Robert H. Shumway • David S. Stoffer Time Series Analysis and Its Applications With R Examples Fourth Edition 123 Robert H. Shumway Department of Statistics University of California, Davis Davis, CA, USA ISSN 1431-875X Springer Texts in Statistics ISBN 978-3-319-52451-1 DOI 10.1007/978-3-319-52452-8 David S. Stoffer Department of Statistics University of Pittsburgh Pittsburgh, PA, USA ISSN 2197-4136 (electronic) ISBN 978-3-319-52452-8 (eBook) Library of Congress Control Number: 2017930675 © Springer International Publishing AG 1999, 2012, 2016, 2017 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Printed on acid-free paper This Springer imprint is published by Springer Nature The registered company is Springer International Publishing AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Preface to the Fourth Edition The fourth edition follows the general layout of the third edition but includes some modernization of topics as well as the coverage of additional topics. The preface to the third edition—which follows—still applies, so we concentrate on the differences between the two editions here. As in the third edition, R code for each example is given in the text, even if the code is excruciatingly long. Most of the examples with seemingly endless coding are in the latter chapters. The R package for the text, astsa, is still supported and details may be found in Appendix R. The global temperature deviation series have been updated to 2015 and are included in the newest version of the package; the corresponding examples and problems have been updated accordingly. Chapter 1 of this edition is similar to the previous edition, but we have included the definition of trend stationarity and the concept of prewhitening when using crosscorrelation. The New York Stock Exchange data set, which focused on an old financial crisis, was replaced with a more current series of the Dow Jones Industrial Average, which focuses on a newer financial crisis. In Chap. 2, we rewrote some of the regression review, changing the smoothing examples from the mortality data example to the Southern Oscillation Index and finding El Niño. We also expanded on the lagged regression example and carried it on to Chap. 3. In Chap. 3, we removed normality from definition of ARMA models; while the assumption is not necessary for the definition, it is essential for inference and prediction. We added a section on regression with ARMA errors and the corresponding problems; this section was previously in Chap. 5. Some of the examples have been modified and we added some examples in the seasonal ARMA section. Finally, we included a discussion of lagged regression with autocorrelated errors. In Chap. 4, we improved and added some examples. The idea of modulated series is discussed using the classic star magnitude data set. We moved some of the filtering section forward for easier access to information when needed. We removed the reliance on spec.pgram (from the stats package) to mvspec (from the astsa package) so we can avoid having to spend pages explaining the quirks of spec.pgram, v vi Preface to the Fourth Edition which tended to take over the narrative. The section on wavelets was removed because there are so many accessible texts available. The spectral representation theorems are discussed in a little more detail using examples based on simple harmonic processes. The general layout of Chap. 5 and of Chap. 7 is the same, although we have revised some of the examples. As previously mentioned, we moved regression with ARMA errors to Chap. 3. Chapter 6 sees the biggest change in this edition. We have added a section on smoothing splines, and a section on hidden Markov models and switching autoregressions. The Bayesian section is completely rewritten and is on linear Gaussian state space models only. The nonlinear material in the previous edition is removed because it was old, and the newer material is in Douc, Moulines, and Stoffer [53]. Many of the examples have been rewritten to make the chapter more accessible. The appendices are similar, with some minor changes to Appendix A and Appendix B. We added material to Appendix C, including a discussion of Riemann– Stieltjes and stochastic integration, a proof of the fact that the spectra of autoregressive processes are dense in the space of spectral densities, and a proof of the fact that spectra are approximately the eigenvalues of the covariance matrix of a stationary process. We tweaked, rewrote, improved, or revised some of the exercises, but the overall ordering and coverage is roughly the same. And, of course, we moved regression with ARMA errors problems to Chap. 3 and removed the Chap. 4 wavelet problems. The exercises for Chap. 6 have been updated accordingly to reflect the new and improved version of the chapter. Davis, CA, USA Pittsburgh, PA, USA December 2016 Robert H. Shumway David S. Stoffer Preface to the Third Edition The goals of this book are to develop an appreciation for the richness and versatility of modern time series analysis as a tool for analyzing data, and still maintain a commitment to theoretical integrity, as exemplified by the seminal works of Brillinger [33] and Hannan [86] and the texts by Brockwell and Davis [36] and Fuller [66]. The advent of inexpensive powerful computing has provided both real data and new software that can take one considerably beyond the fitting of simple time domain models, such as have been elegantly described in the landmark work of Box and Jenkins [30]. This book is designed to be useful as a text for courses in time series on several different levels and as a reference work for practitioners facing the analysis of time-correlated data in the physical, biological, and social sciences. We have used earlier versions of the text at both the undergraduate and graduate levels over the past decade. Our experience is that an undergraduate course can be accessible to students with a background in regression analysis and may include Sects. 1.1–1.5, Sects. 2.1–2.3, the results and numerical parts of Sects. 3.1–3.9, and briefly the results and numerical parts of Sects. 4.1–4.4. At the advanced undergraduate or master’s level, where the students have some mathematical statistics background, more detailed coverage of the same sections, with the inclusion of extra topics from Chaps. 5 or 6, can be used as a one-semester course. Often, the extra topics are chosen by the students according to their interests. Finally, a two-semester upper-level graduate course for mathematics, statistics, and engineering graduate students can be crafted by adding selected theoretical appendices. For the upper-level graduate course, we should mention that we are striving for a broader but less rigorous level of coverage than that which is attained by Brockwell and Davis [36], the classic entry at this level. The major difference between this third edition of the text and the second edition is that we provide R code for almost all of the numerical examples. An R package called astsa is provided for use with the text; see Sect. R.2 for details. R code is provided simply to enhance the exposition by making the numerical examples reproducible. vii viii Preface to the Third Edition We have tried, where possible, to keep the problem sets in order so that an instructor may have an easy time moving from the second edition to the third edition. However, some of the old problems have been revised and there are some new problems. Also, some of the data sets have been updated. We added one section in Chap. 5 on unit roots and enhanced some of the presentations throughout the text. The exposition on state-space modeling, ARMAX models, and (multivariate) regression with autocorrelated errors in Chap. 6 have been expanded. In this edition, we use standard R functions as much as possible, but we use our own scripts (included in astsa) when we feel it is necessary to avoid problems with a particular R function; these problems are discussed in detail on the website for the text under R Issues. We thank John Kimmel, Executive Editor, Springer Statistics, for his guidance in the preparation and production of this edition of the text. We are grateful to Don Percival, University of Washington, for numerous suggestions that led to substantial improvement to the presentation in the second edition, and consequently in this edition. We thank Doug Wiens, University of Alberta, for help with some of the R code in Chaps. 4 and 7, and for his many suggestions for improvement of the exposition. We are grateful for the continued help and advice of Pierre Duchesne, University of Montreal, and Alexander Aue, University of California, Davis. We also thank the many students and other readers who took the time to mention typographical errors and other corrections to the first and second editions. Finally, work on this edition was supported by the National Science Foundation while one of us (D.S.S.) was working at the Foundation under the Intergovernmental Personnel Act. Davis, CA, USA Pittsburgh, PA, USA September 2010 Robert H. Shumway David S. Stoffer Contents Preface to the Fourth Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Preface to the Third Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii 1 Characteristics of Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 The Nature of Time Series Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Time Series Statistical Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Measures of Dependence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Stationary Time Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Estimation of Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Vector-Valued and Multidimensional Series . . . . . . . . . . . . . . . . . . . . . Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 8 15 19 26 33 38 2 Time Series Regression and Exploratory Data Analysis . . . . . . . . . . . . . 2.1 Classical Regression in the Time Series Context . . . . . . . . . . . . . . . . . 2.2 Exploratory Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Smoothing in the Time Series Context . . . . . . . . . . . . . . . . . . . . . . . . . Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 45 54 65 70 3 ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Autoregressive Moving Average Models . . . . . . . . . . . . . . . . . . . . . . . 3.2 Difference Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Autocorrelation and Partial Autocorrelation . . . . . . . . . . . . . . . . . . . . . 3.4 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Integrated Models for Nonstationary Data . . . . . . . . . . . . . . . . . . . . . . 3.7 Building ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Regression with Autocorrelated Errors . . . . . . . . . . . . . . . . . . . . . . . . 3.9 Multiplicative Seasonal ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 75 88 94 100 113 131 135 142 145 154 ix x Contents 4 Spectral Analysis and Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Cyclical Behavior and Periodicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 The Spectral Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Periodogram and Discrete Fourier Transform . . . . . . . . . . . . . . . . . . . 4.4 Nonparametric Spectral Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Parametric Spectral Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Multiple Series and Cross-Spectra . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Linear Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Lagged Regression Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 Signal Extraction and Optimum Filtering . . . . . . . . . . . . . . . . . . . . . . . 4.10 Spectral Analysis of Multidimensional Series . . . . . . . . . . . . . . . . . . . Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 166 172 179 189 203 206 211 217 222 226 229 5 Additional Time Domain Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Long Memory ARMA and Fractional Differencing . . . . . . . . . . . . . . 5.2 Unit Root Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 GARCH Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Threshold Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Lagged Regression and Transfer Function Modeling . . . . . . . . . . . . . 5.6 Multivariate ARMAX Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 241 250 253 262 266 272 285 6 State Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Linear Gaussian Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Filtering, Smoothing, and Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Maximum Likelihood Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Missing Data Modifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Structural Models: Signal Extraction and Forecasting . . . . . . . . . . . . 6.6 State-Space Models with Correlated Errors . . . . . . . . . . . . . . . . . . . . . 6.6.1 ARMAX Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6.2 Multivariate Regression with Autocorrelated Errors . . . . . . . 6.7 Bootstrapping State Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.8 Smoothing Splines and the Kalman Smoother . . . . . . . . . . . . . . . . . . . 6.9 Hidden Markov Models and Switching Autoregression . . . . . . . . . . . 6.10 Dynamic Linear Models with Switching . . . . . . . . . . . . . . . . . . . . . . . 6.11 Stochastic Volatility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.12 Bayesian Analysis of State Space Models . . . . . . . . . . . . . . . . . . . . . . Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 290 294 304 313 318 321 323 324 328 333 336 348 360 367 378 7 Statistical Methods in the Frequency Domain . . . . . . . . . . . . . . . . . . . . . 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Spectral Matrices and Likelihood Functions . . . . . . . . . . . . . . . . . . . . 7.3 Regression for Jointly Stationary Series . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Regression with Deterministic Inputs . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Random Coefficient Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385 385 388 390 399 407 Contents xi 7.6 Analysis of Designed Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7 Discriminant and Cluster Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8 Principal Components and Factor Analysis . . . . . . . . . . . . . . . . . . . . . 7.9 The Spectral Envelope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409 423 439 455 466 Appendix A Large Sample Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.1 Convergence Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Central Limit Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.3 The Mean and Autocorrelation Functions . . . . . . . . . . . . . . . . . . . . . . 473 473 480 484 Appendix B Time Domain Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.1 Hilbert Spaces and the Projection Theorem . . . . . . . . . . . . . . . . . . . . . B.2 Causal Conditions for ARMA Models . . . . . . . . . . . . . . . . . . . . . . . . . B.3 Large Sample Distribution of the AR Conditional Least Squares Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.4 The Wold Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493 493 497 Appendix C Spectral Domain Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.1 Spectral Representation Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.2 Large Sample Distribution of the Smoothed Periodogram . . . . . . . . . C.3 The Complex Multivariate Normal Distribution . . . . . . . . . . . . . . . . . C.4 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.4.1 Riemann–Stieltjes Integration . . . . . . . . . . . . . . . . . . . . . . . . . . C.4.2 Stochastic Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.5 Spectral Analysis as Principal Component Analysis . . . . . . . . . . . . . . C.6 Parametric Spectral Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505 505 509 519 524 524 526 528 531 Appendix R R Supplement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . R.1 First Things First . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . R.2 ast...
View Full Document

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture