RidgeRegrAndLasso_PenalizationMethodsContinued.pdf - Ridge Regression(an L2 penalty approach Consider y = X where X is N(p 1 real matrix with full

# RidgeRegrAndLasso_PenalizationMethodsContinued.pdf - Ridge...

• 8
• 100% (1) 1 out of 1 people found this document helpful

This preview shows page 1 - 4 out of 8 pages.

The Lasso (an L 1 penalty approach). LASSO stands for "Least Absolute Shrinkage And Selection Operator" (Tibshirani, 1996). ˆ β Lasso = argmin β N X i =1 y i - β 0 - p X j =1 x ij β j ! 2 + λ p X j =1 | β j | or equivalently ˆ β Lasso = argmin β N X i =1 y i - β 0 - p X j =1 x ij β j ! 2 subject to p X j =1 | β j | ≤ t.
Elements of Statistical Learning (2nd Ed.) c Hastie, Tibshirani & Friedman 2009 Chap 3 β ^ β ^ 2 . . β 1 β 2 β 1 β FIGURE 3.11. Estimation picture for the lasso (left) and ridge regression (right). Shown are contours of the error and constraint functions. The solid blue areas are the constraint regions | β 1 | + | β 2 | ≤ t and β 2 1 + β 2 2