# jan19 - STA 414/2104 Jan 19, 2010 Administration Homework 1...

This preview shows pages 1–8. Sign up to view the full content.

STA 414/2104 Jan 19, 2010 Administration I Homework 1 available Thursday I Discussion of project requirements on Thursday I NSERC summer undergraduate awards I Fields-MITACS undergraduate summer research http://www.fields.utoronto.ca/programs/scientific/10-11/summer-research/ 1 / 24

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
STA 414/2104 Jan 19, 2010 Geometric view of least squares ﬁtting I ˆ β = ( X T X ) - 1 X T y I ˆ β = ( ˆ β 0 , ˆ β 1 , . . . , ˆ β p ) I ˆ β p can be obtained by a series of regressions (projections) as outlined in algorithm 3.1 on p.54 regress x 1 on 1, get coefﬁcient ˆ γ 01 , form residual z 1 = x 1 - ˆ x 1 regress x 2 on 1, z 1 , get coefs ˆ γ 02 , ˆ γ 12 , form residual z 2 = x 2 - ˆ γ 02 1 - ˆ γ 12 z 1 . . . regress x p on z p - 1 , z p - 2 , . . . , z 1 , 1 to get z p = x p - ˆ x p regress y on z p to get ˆ β p I illustration on prostate training data – see prostateRsession.txt 2 / 24
STA 414/2104 Jan 19, 2010 QR Decomposition X = Z Γ I matrix representation X = Z Γ I Z has columns z j I Γ = 0 ˆ γ 01 ˆ γ 02 . . . ˆ γ 0 p 0 0 ˆ γ 12 . . . ˆ γ 1 p . . . ˆ γ p - 1 , p 0 . . . 0 I Let D jj = ( z T j z j ) 1 / 2 = || z j || and D = diag ( D jj ) dimension? I X = ZD - 1 D Γ = QR I Q T Q = I , R is upper triangular I ˆ β = R - 1 Q T y I ˆ y = QQ T y check pr.lm\$qr I Gram-Schmidt 3 / 24

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
STA 414/2104 Jan 19, 2010 Singular value decomposition, X = UDV I X assumed to be centered, so all columns add to zero (p.64, l.-4) I U is N × p , V is p × p I D is diagonal, d 1 d 2 ≥ ··· ≥ d p 0 not the same D I X T X = VD 2 V T eigendecomposition of X T X and of NS = XX T I ˆ y = X ˆ β = UU T y = QQ T y different orthogonal bases for . .. I deﬁne z 1 = Xv 1 = u 1 d 1 I note that var ( z 1 ) = d 2 1 / N I z 1 is the derived variable with the largest variance: the ﬁrst principal component of X I z 2 has the largest variance among linear combinations orthogonal to z 1 4 / 24

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
STA 414/2104 Jan 19, 2010 ... singular value decomposition I X N × p = U N × p D p × p V T p × p , U T U = I , V T V = I , D = diag ( d 1 , . . . d p ) X ˆ β LS = X ( X T X ) - 1 X T y = UDV T ( VD U T U DV T ) - 1 VDU T y = UD V T V T - 1 D - 2 V - 1 V DU T y = UU T y = Σ p j = 1 u j u T j y svd(model.matrix(pr.lm)) , for example 6 / 24
STA 414/2104 Jan 19, 2010 Model selection: subsets ( § 3.3.1) I linear regression: forward, backward, stepwise, all possible subsets regression I RSS ( ˆ β ) = ( y - X ˆ β ) T ( y - X ˆ β

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This document was uploaded on 08/12/2010.

### Page1 / 25

jan19 - STA 414/2104 Jan 19, 2010 Administration Homework 1...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online