recitation4 Logistic Regression

recitation4 Logistic Regression - Recitation 10-701/15-781,...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Recitation 10-701/15-781, Fall 2008 Logistic Regression By Hanghang Tong (htong@cs) Some Parts of the slides are from previous years’ recitation and lecture notes 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Why (binary) LR is a linear classifier Gradient Ascent How & What is the intuition Over-fitting and how to overcome LR vs. NB 2
Background image of page 2
LR: Binary Case Learn P(Y|X) directly Logistic regression for binary classification: Note: Generative classifier: learn P(X|Y), P(Y) to get P(Y|X) under some modeling assumption e.g. P(X|Y) ~ N(m y , 1), etc. i i i X w w w X Y P )) ( exp( 1 1 ) , | 1 ( 0 i i i i i i X w w X w w w X Y P )) ( exp( 1 )) ( exp( ) , | 0 ( 0 0 3
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Why binary LR is a linear classifier For a given X, we want to know when P(Y=1|X,w) ≥P(Y=0|X,w)? P(Y=1|X, w) >= 0.5 0 1 0.5 1 exp( ( )) ii i w w X 0 exp( ( )) 1 i w w X 0 0 i w w X  4
Background image of page 4
Gradient ascent for LR (Recap) Iterate until change < threshold For all i , )) , | 1 ( ( ) ( ))) ( exp( 1 ln( ) )( 1 ( ) , | ( ln ) ( 1 1 0 0 w X Y P Y X w w l X w w X w w Y w X Y P w l l l l l l i i l n i n i l i i l i i l l l l
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/26/2010 for the course MACHINE LE 10701 taught by Professor Ericp.xing during the Fall '08 term at Carnegie Mellon.

Page1 / 18

recitation4 Logistic Regression - Recitation 10-701/15-781,...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online