jan26 - STA 414/2104 Administration Homework 1 on web page...

Info iconThis preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon
STA 414/2104 Jan 26, 2010 Administration I Homework 1 on web page, due Feb 11 I NSERC summer undergraduate award applications due Feb 5 I Some helpful books 1 / 35
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
STA 414/2104 Jan 26, 2010 ... administration 2 / 35
Background image of page 2
STA 414/2104 Jan 26, 2010 ... administration I collection of tools for regression and classification I some old (least squares, discriminant analysis) I some new (lasso, support vector machines) I statistical justifications: loss, likelihood, mean squared error, classification error, posterior probabilities. .. I statistical thinking I framework for analysing new tools 3 / 35
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
STA 414/2104 Jan 26, 2010 Linear regression plus I variable selection: forward, backward, stepwise, all possible subsets I comparing models: adjusted R 2 , C p vs. p , AIC 1 and variants, K -fold cross-validation I shrinkage methods: ridge regression, lasso, Least Angle Regression I tuning parameter (amount of shrinkage): validation data, K -fold cross-validation I derived variables: principal components regression, partial least squares 1 C p and AIC can be shown to be estimates of expected prediction error 4 / 35
Background image of page 4
STA 414/2104 Jan 26, 2010 Predictions with smoothing regression I on training data: x ’s are centered and scaled when fitting Lasso, LAR, and ridge regression I ˆ β 0 is not included in shrinkage I (3.41) ˆ β ridge = argmin β X ( y i - β 0 - p X j = 1 x ij β j ) 2 + λ p X j = 1 β 2 j I (3.52) ˆ β lasso = argmin β X ( y i - β 0 - p X j = 1 x ij β j ) 2 + λ p X j = 1 | β j | I ˆ β 0 = ¯ y = ¯ y train I for predicting a new y 0 = ˆ β 0 + p j = 1 ( x 0 j - ¯ x j ) ˆ β j : use ˆ β 0 = ¯ y train , and ¯ x j means average for j th feature on the training data I see construction of tx and mm in Table33R.txt 5 / 35
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
STA 414/2104 Jan 26, 2010 ...predictions > options(digits=4) > as.vector(predict.lars(pr.lars,newx=test[,1:8],type="fit",s=0.36, mode="fraction")$fit) [1] 2.094 1.443 1.780 2.292 2.695 2.009 2.283 1.731 1.969 1.694 2.588 2.486 [13] 2.712 2.492 2.554 2.345 2.053 2.954 3.189 1.976 2.946 3.035 2.746 2.550 [25] 2.636 2.697 3.135 3.095 3.233 3.698 > as.vector(tx % * % coef(pr.lars,s=0.36,mode="fraction"))+mean(train$lpsa) [1] 2.094 1.443 1.780 2.292 2.695 2.009 2.283 1.731 1.969 1.694 2.588 2.486 [13] 2.712 2.492 2.554 2.345 2.053 2.954 3.189 1.976 2.946 3.035 2.746 2.550 [25] 2.636 2.697 3.135 3.095 3.233 3.698 6 / 35
Background image of page 6
Derived features § 3.5 I replace x 1 ,... x p with linear combinations of columns I principal components from SVD are natural candidates I X = UDV T , U T U = I N , V T V = I p I z m = Xv m , m = 1 ,..., M < p I z m are orthogonal by construction I ˆ y pcr ( M ) = ¯ y 1 + M X m = 1 ˆ θ m z m I ˆ θ m = h z m , y i h z m , z m i = n i = 1 z mi y i n i = 1 z 2 mi I inputs should be scaled first (mean 0, variance 1) I Angle brackets notation explained at (3.25). Exercise:
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 08/12/2010.

Page1 / 35

jan26 - STA 414/2104 Administration Homework 1 on web page...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online