Chap4-Part2 - Machine Learning Srihari Linear...

Info iconThis preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Linear Classification: Probabilistic Generative Models Sargur N. Srihari University at Buffalo, State University of New York USA Machine Learning Srihari
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Linear Classification using Probabilistic Generative Models • Topics 1. Overview (Generative vs Discriminative) 2. Bayes Classifier • using Logistic Sigmoid and Softmax 3. Continuous inputs • Gaussian Distributed Class-conditionals – Parameter Estimation 4. Discrete Features 5. Exponential Family 2 Machine Learning Srihari
Background image of page 2
Machine Learning Srihari 3 Overview of Methods for Classification 1. Generative Models ( Two-step) 1. Infer class-conditional densities p (x| C k ) and priors p(C k ) 2. Use Bayes theorem to determine posterior probabilities 2. Discriminative Models ( One-step) Directly infer posterior probabilities p(C k | x ) Decision Theory In both cases use decision theory to assign each new x to a class
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Generative Model • Model class conditionals p (x| C k ) and priors p ( C k ) • Compute posteriors p ( C k |x) from Bayes theorem • Two class Case • Posterior for class C 1 is 4 Since LLR with Bayes odds Machine Learning Srihari p (x) = p (x, C i ) i = p (x/ C i ) i p ( C i )
Background image of page 4
Logistic Sigmoid Function Sigmoid: “S”-shaped or squashing function maps real a ε (- , + ) to finite (0,1) interval Note: Dotted line is scaled probit function cdf of a zero-mean unit variance Gaussian σ ( a ) = 1 1 + exp( a ) Property : ( a ) = 1 −σ ( a ) Inverse : a = ln 1 −σ If then Inverse represents ln[ p ( C 1 |x)/ p ( C 2 |x) Log ratio of probabilities called logit or log odds a Machine Learning 5 Srihari ( a ) ( a ) = P ( C 1 | x)
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
• More than 2 classes • Gaussian Distribution of x • Discrete Features • Exponential Family Machine Learning Srihari 6
Background image of page 6
Image of page 7
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 18

Chap4-Part2 - Machine Learning Srihari Linear...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online