elastic net - Regularization and Variable Selection via the...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Regularization and Variable Selection via the Elastic Net Hui Zou and Trevor Hastie Journal of Royal Statistical Society, B, 2005 Presenter: Minhua Chen, Nov. 07, 2008 – p. 1/1 7 Agenda Introduction to Regression Models. Motivation for Elastic Net. Naive Elastic Net and its grouping effect. Elastic Net. Experiments and Conclusions. – p. 2/1 7 Introduction to Regression Models Consider the following regression model with p predictors and n samples: y = Xβ + ǫ where X n × p = [ x 1 , x 2 , ··· , x p ] , β = [ β 1 , β 2 , ··· , β p ] ⊤ and y = [ y 1 , y 2 , ··· , y n ] ⊤ . ǫ is the additive noise with dimension n × 1 . Suppose the predictors ( x i ) are normalized to mean zero and variance one, and the regression output y sums to zero. Ordinary Least Squares (OLS): ˆ β ( OLS ) = arg min β bardbl y − Xβ bardbl 2 Ridge Regression: ˆ β ( Ridge ) = arg min β bardbl y − Xβ bardbl 2 + λ bardbl β bardbl 2 LASSO: ˆ β ( LASSO ) = arg min β bardbl y − Xβ bardbl 2 + λ | β | 1 ( | β | 1 defines p summationdisplay j =1 | β j | ) Elastic Net: ˆ β ( Naive ENet ) = arg min β bardbl y − Xβ bardbl 2 + λ 1 | β | 1 + λ 2 bardbl β bardbl 2 ˆ β ( ENet ) = (1 + λ 2 ) · ˆ β ( Naive ENet ) (1) – p. 3/1 7 Motivation for Elastic Net Prediction accuracy and model interpretation are two important aspects of regression models. LASSO is a penalized regression method to improve OLS and Ridge regression. LASSO does shrinkage and variable selection simultaneously for better prediction and model interpretation. Disadvantage of LASSO: LASSO selects at most n variables before it saturates....
View Full Document

This note was uploaded on 02/27/2012 for the course STATS 315A taught by Professor Tibshirani,r during the Spring '10 term at Stanford.

Page1 / 17

elastic net - Regularization and Variable Selection via the...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online