Oo o oo o ooo o o oooo o o o oooo o o o o oo o oo o o

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Newton algorithm ∂ (β ) ∂β ∂ 2 (β ) ∂β∂β T = X T ( y − p) = −XT WX. A Newton step is thus β new = β old + (XT WX)−1 XT (y − p) = (XT WX)−1 XT W Xβ old + W−1 (y − p) = (XT WX)−1 XT Wz. In the second and third line we have re-expressed the Newton-Raphson step as a weighted least squares step, with the response z = Xβ old + W−1 (y − p). 42 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani Properties of logistic regression solutions ˆ • Satisfy score equations XT (y − p) = 0. • If W is a diagonal matrix with weights wi = pi (1 − pi ), then the ˆ ˆ ˆ asymptotic covariance matrix of β is ˆ cov(β ) = (XT WX)−1 • If the two classes are linearly separable, then the solution is undefined! [MLE tries to achieve probabilities that are 0 and 1, and ˆ for this some of β must go to ±∞]. • Inference proceeds in a manner very similar to that for linear regression. • Sometimes called a forward or discriminative model; versus LDA which is backwards or generative. 43 0.4 0.8 o oo o oo o o o oo oo o oo ooo oo o o o oooo o o oo o oooo ooo oooooooooo o ooooo o o ooo o ooo o oo o ooooooo o ooooooooooo oo o o oooooooooo oo ooooooooooo oooo o oo o o ooooooooooooo o o ooooooo o oooooooo ooo ooooooooooooooo oo o ooooo oooooo o ooooooo o oo ooo tobacco ldl 50 100 o oo oo o o o o oo o o oo o o o oo oo o oo o o oo ooo o o oo o ooo oo oo o oooo o oooo o o ooo oo ooo ooo oo o oooo oooooo o o o oo ooo ooo o oooo o o ooooooooo oo o oooo oo o oo oo oooooo ooo o o o oo o oooooooooo oo o oo oo o oo oooooo ooo o o ooo ooo o o oo o o oo oo o oo o o oo o o oo oooooooooooo oo ooo o oooooooooo o o ooo oo o oooooooooooooo oo o o oo ooo o o oo oooooo o oooooooo o ooo ooooooooo oo ooooooooo oo o o oooooooooooooooooo oooo oo oooooooooo ooooo o o ooo o oo oo oo oo o o oo o o o o ooooooo o oo o ooo oooooo ooo oo o oo ooooo ooooooooooo o oooo o o oooo ooo o oo oo oooo o o oo oo o oo oo o o o o oooo ooo o o ooo ooo o oooo ooo o oo o oooo o oo o o oo oo o o o o o o o oo o o o o o o oo oo o o oo o oo o oo o ooo o o oooo o o o oooo o o o o oo o oo o o oo o oooooooo o o o o o o ooooo o o o o oo o o oo o oo ooooo oooooo o oooooooooooo o o oo o o o ooooooo o ooooo o ooo ooo o...
View Full Document

This document was uploaded on 03/10/2014 for the course STATS 315A at Stanford.

Ask a homework question - tutors are online