This preview shows pages 1–8. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: STA 414/2104 S: February 2 2010 Administration I HW due February 11 by 1 pm I Thursday, February 4: TA Li Li has office hours 12 (in class), 23 in SS 6027A I Chapter 3: 3.1, 3.2 (except 3.2.4), 3.3 (except 3.3.3), 3.4 (except 3.4.4), 3.5.1 I Chapter 4: 4.1, 4.2, 4.3 (except 4.3.1, 4.3.2), 4.4.0, 4.4.1, 4.4.2 I My office hours are Tuesday 34 and Thursday 23 (although cancelled on Thursday Feb 4) I http://www3.interscience.wiley.com/cgibin/ fulltext/123233977/HTMLSTART 1/30 STA 414/2104 S: February 2 2010 A few points on logistic regression I Logistic regression: Pr ( G = k  X ) linear on the logit scale I Linear discriminant analysis: Pr ( G = k  X ) Pr ( X  G = k ) Pr ( G = k ) I Theory: LDA more efficient if X really is normal I Practice: LR usually viewed as more robust, but HTF claim prediction errors are very similar I See: last slide of Jan 26 for calculation of prediction errors on training data I Data: LR is more complicated with K > 2; use multinom in the MASS library I Lasso version of logistic regression described in 4.4.4 2/30 STA 414/2104 S: February 2 2010 ... logistic regression I Deviance in a generalized linear model (such as LR), is 2 log L ( ) + constant I Comparing deviances from two model fits is a loglikelihood ratio test that the corresponding parameters are 0 I AIC compares instead 2 log L ( ) + 2 p I for Binomial data, but not for binary data, residual deviance provides a test of goodness of fit of the binomial model 3/30 STA 414/2104 S: February 2 2010 Flexible modelling using basis expansions (Chapter 5) I Linear regression: y = X + , ( , 2 ) I Smooth regression: y = f ( X ) + I f ( X ) = E ( Y  X ) to be specified I Flexible linear modelling f ( X ) = M m = 1 m h m ( X ) I This is called a linear basis expansion , and h m is the m th basis function I For example if X is onedimensional: f ( X ) = + 1 X + 2 X 2 , or f ( X ) = + 1 sin ( X ) + 2 cos ( X ) , etc. I Simple linear regression has h 1 ( X ) = 1, h 2 ( X ) = X . Several other examples on p.140 4/30 STA 414/2104 S: February 2 2010 I Polynomial fits: h j ( x ) = x j , j = ,..., m I Fit using linear regression with design matrix X , where X ij = h j ( x i ) I Justification is that any smooth function can be approximated by a polynomial expansion (Taylor series) I Can be difficult to fit numerically, as correlation between columns can be large I May be useful locally, but less likely to work over the range of X I Idea: fit polynomials locally in X I Need to be careful not to overfit, since we are using only a fraction of the data 5/30 STA 414/2104 S: February 2 2010 Piecewise polynomials I piecewise constant basis functions h 1 ( X ) = I ( X < 1 ) , h 2 ( X ) = I ( 1 X < 2 ) , h 3 ( x ) = I ( 2 X ) I fitting by local averaging I piecewise linear basis functions , with constraints h 1 ( X ) = 1 , h 2 ( X ) = X h 3 ( X ) = ( X 1 ) + , h 4 ( X ) = ( X 2 ) + I windows defined by knots 1 , 2 ,......
View
Full
Document
This document was uploaded on 08/12/2010.
 Spring '09

Click to edit the document details