jan19 - STA 414/2104 Jan 19, 2010 Administration Homework 1...

Info iconThis preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon
STA 414/2104 Jan 19, 2010 Administration I Homework 1 available Thursday I Discussion of project requirements on Thursday I NSERC summer undergraduate awards I Fields-MITACS undergraduate summer research http://www.fields.utoronto.ca/programs/scientific/10-11/summer-research/ 1 / 24
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
STA 414/2104 Jan 19, 2010 Geometric view of least squares fitting I ˆ β = ( X T X ) - 1 X T y I ˆ β = ( ˆ β 0 , ˆ β 1 , . . . , ˆ β p ) I ˆ β p can be obtained by a series of regressions (projections) as outlined in algorithm 3.1 on p.54 regress x 1 on 1, get coefficient ˆ γ 01 , form residual z 1 = x 1 - ˆ x 1 regress x 2 on 1, z 1 , get coefs ˆ γ 02 , ˆ γ 12 , form residual z 2 = x 2 - ˆ γ 02 1 - ˆ γ 12 z 1 . . . regress x p on z p - 1 , z p - 2 , . . . , z 1 , 1 to get z p = x p - ˆ x p regress y on z p to get ˆ β p I illustration on prostate training data – see prostateRsession.txt 2 / 24
Background image of page 2
STA 414/2104 Jan 19, 2010 QR Decomposition X = Z Γ I matrix representation X = Z Γ I Z has columns z j I Γ = 0 ˆ γ 01 ˆ γ 02 . . . ˆ γ 0 p 0 0 ˆ γ 12 . . . ˆ γ 1 p . . . ˆ γ p - 1 , p 0 . . . 0 I Let D jj = ( z T j z j ) 1 / 2 = || z j || and D = diag ( D jj ) dimension? I X = ZD - 1 D Γ = QR I Q T Q = I , R is upper triangular I ˆ β = R - 1 Q T y I ˆ y = QQ T y check pr.lm$qr I Gram-Schmidt 3 / 24
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
STA 414/2104 Jan 19, 2010 Singular value decomposition, X = UDV I X assumed to be centered, so all columns add to zero (p.64, l.-4) I U is N × p , V is p × p I D is diagonal, d 1 d 2 ≥ ··· ≥ d p 0 not the same D I X T X = VD 2 V T eigendecomposition of X T X and of NS = XX T I ˆ y = X ˆ β = UU T y = QQ T y different orthogonal bases for . .. I define z 1 = Xv 1 = u 1 d 1 I note that var ( z 1 ) = d 2 1 / N I z 1 is the derived variable with the largest variance: the first principal component of X I z 2 has the largest variance among linear combinations orthogonal to z 1 4 / 24
Background image of page 4
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
STA 414/2104 Jan 19, 2010 ... singular value decomposition I X N × p = U N × p D p × p V T p × p , U T U = I , V T V = I , D = diag ( d 1 , . . . d p ) X ˆ β LS = X ( X T X ) - 1 X T y = UDV T ( VD U T U DV T ) - 1 VDU T y = UD V T V T - 1 D - 2 V - 1 V DU T y = UU T y = Σ p j = 1 u j u T j y svd(model.matrix(pr.lm)) , for example 6 / 24
Background image of page 6
STA 414/2104 Jan 19, 2010 Model selection: subsets ( § 3.3.1) I linear regression: forward, backward, stepwise, all possible subsets regression I RSS ( ˆ β ) = ( y - X ˆ β ) T ( y - X ˆ β
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 08/12/2010.

Page1 / 25

jan19 - STA 414/2104 Jan 19, 2010 Administration Homework 1...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online