This linearity is a consequence of the gaussian

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: on-null False Neg. (FN) True Pos. (TP) P Total N∗ P∗ Possible results when applying a classifier or diagnostic test to a population. 52 ESL Chapter 4 — Linear Methods for Classification Name Trevor Hastie and Rob Tibshirani Definition Synonyms False Pos. Rate FP/N Type I Error, 1−Specificity True Pos. Rate TP/P 1−Type II Error, Power, Sensitivity, Recall Pos. Pred. Value TP/P∗ Neg. Pred. Value TN/N∗ Precision, 1−False Discovery Proportion Important measures for classification and diagnostic testing 53 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani Multiple Logistic Regression T Model is defined in terms of J − 1 logits ηj (X ) = βj X : P (G = 1|X ) P (G = J | X ) P (G = 2|X ) log P (G = J | X ) log = η1 (X ) = η2 (X ) . . . log P (G = J − 1|X ) P (G = J | X ) P (G = j | X ) = = ηJ −1 (X ) eηj (X ) 1+ J − 1 η (X ) =1 e Fit by least squares or multinomial maximum likelihood. 54 ESL Chapter 4 — Linear Methods for Classification Logistic Regression with p Trevor Hastie and Rob Tibshirani N • Typically linear models are sufficient — logit(pi ) = β T xi • Models have to be regularized – ridge penalty — similar to SVM N {yi log pi + (1 − yi ) log(1 − pi )} − λ||β ||2 PLL = i=1 – lasso penalty — selects variables p N |βj | {yi log pi + (1 − yi ) log(1 − pi )} − λ PLL = i=1 j =1 • IRLS algorithm for ridge, and LARS-like algorithm for Lasso 55 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani Glmnet software in R Glmnet fits the GLM family of models by penalized maximum likelihood. This includes (multiple) logistic regression. Glmnet computes the entire “regularization path” for the “elastic net” penalty family: 1 max l(β ) − λ (1 − α)||β ||2 + α||β ||1 2 β 2 • The regularization path follows a complete grid of values for λ, with α fixed. • α spans ridge to lasso • For multiple logistic regression, the model is symmetric...
View Full Document

This document was uploaded on 03/10/2014 for the course STATS 315A at Stanford.

Ask a homework question - tutors are online