statlearn

# statlearn - Notes on Statistical Learning John I. Marden...

This preview shows pages 1–5. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Notes on Statistical Learning John I. Marden Copyright 2006 2 Contents 1 Introduction 5 2 Linear models 7 2.1 Good predictions: Squared error loss and in-sample error . . . . . . . . . . . 8 2.2 Matrices and least-squares estimates . . . . . . . . . . . . . . . . . . . . . . 9 2.3 Mean vectors and covariance matrices . . . . . . . . . . . . . . . . . . . . . . 10 2.4 Prediction using least-squares . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.5 Subset selection and Mallows’ C p . . . . . . . . . . . . . . . . . . . . . . . . 13 2.5.1 Estimating the in-sample errors . . . . . . . . . . . . . . . . . . . . . 15 2.5.2 Finding the best subset . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.5.3 Using R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.6 Regularization: Ridge regression . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.6.1 Estimating the in-sample errors . . . . . . . . . . . . . . . . . . . . . 24 2.6.2 Finding the best λ . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.6.3 Using R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.7 Lasso . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.7.1 Estimating the in-sample errors . . . . . . . . . . . . . . . . . . . . . 33 2.7.2 Finding the best λ . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 2.7.3 Using R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3 Linear Predictors of Non-linear Functions 39 3.1 Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.1.1 Leave-one-out cross-validation . . . . . . . . . . . . . . . . . . . . . . 48 3.1.2 Using R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.1.3 The cross-validation estimate . . . . . . . . . . . . . . . . . . . . . . 55 3.2 Sines and cosines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.2.1 Estimating σ 2 e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 3.2.2 Cross-validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.2.3 Using R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.3 Local fitting: Regression splines . . . . . . . . . . . . . . . . . . . . . . . . . 70 3.3.1 Using R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.4 Smoothing splines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 3 4 CONTENTS 3.4.1 Using R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 3.4.2 An interesting result . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 3.5 A glimpse of wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 3.5.1 Haar wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 3.5.2 An example of another set of wavelets . . . . . . . . . . . . . . . . . 91 3.5.3 Example Using R . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Example Using R ....
View Full Document

## This note was uploaded on 02/27/2012 for the course STATS 315A taught by Professor Tibshirani,r during the Spring '10 term at Stanford.

### Page1 / 123

statlearn - Notes on Statistical Learning John I. Marden...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online