Y 01 codes the classes model px p r y 1x logitpx

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ds of earlier slide. 37 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani 0.7 • 0.5 • • • • • • • • • • • Test Data Train Data 0.4 Misclassification Rate • 0.6 LDA and Dimension Reduction on the Vowel Data • • 0.3 • • • 2 4 6 • • 8 10 Dimension Training and test error rates for the vowel data, as a function of the dimension of the discriminant subspace. In this case the best error rate is for dimension 2. 38 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani Performance on Vowel Data Train Test Linear Regression 0.48 0.67 LDA 0.32 0.56 Reduced Rank LDA 0.36 0.50 QDA 0.01 0.53 Logistic Regression 0.22 0.51 39 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani Classification in Reduced Subspace o oo o oo oo o oo o o oo o o oo oo o o oo oo oo oo o oo oo o oo o oo oo o o o o oo o oo o o oo o oo o • oo o oo oo o o o oo oo • ooo o o o ooo o oo o o oo o ooo o o o o o o o o ooo o o o o o o o o o • o o oooo o o o o o o oo o oo o o oo o oo oo oo oo o•oo o o ooo o o o o o o oo oo o o o o o o o oo oo o oo o o o oo o oo o o o oo o oo o ooo o o oo o o o o oo ooo ooo o oo ooo ooo o o • o o oo o o ooo o o o oo oo o oo o • ooo o o o oo o o o o ooo o ooo o oo oo o oo o o o o oo oo o o • o oo o o o o o o o oo oo o o o oo oo o o o o o oo o o oo oo oooo o oo o o oo oo o o oo o oo o oo o o o o oo o o oo oo o o o oooo o o o oo o o o oo o oo o• o o oo o o o o o oo o o oo oo o o o o o• o o ooo o o oo o o ooo o o o o o oo ooo • o o o oo oo o o o o oo oo o o oo o o oo oo o o o o oo o oo o o oo oo oo oo oo o o o o• o oo o o o o oo o oo o oo o o oo o oo o o o o o o Canonical Coordinate 2 • • • • • • • • • • • oo o o o o Canonical Coordinate 1 40 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani Linear Logistic Regression Two-class case. Y = 0/1 codes the classes. Model p(x) = P r (Y = 1|x). logitp(x) ≡ log p(x) = β T x, 1 − p(x) βT x p(x) = e 1 + eβ T x n {yi log pi + (1 − yi ) log(1 − pi )} Log-Likelihood = i=1 IRLS algorithm 1. Initialize β . 2. Form linearized responses zi = β T xi + (yi − pi )/{pi (1 − pi )} 3. Form weights wi = pi (1 − pi ) 4. Update β by weighted LS of zi on xi with weights wi . Steps 2-4 are repeated until convergence. 41 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani IRLS...
View Full Document

This document was uploaded on 03/10/2014 for the course STATS 315A at Stanford.

Ask a homework question - tutors are online