Lecture 16(a) - Linear Discriminant Analysis

# Lecture 16(a) - Linear Discriminant Analysis - Pattern...

This preview shows pages 1–12. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Pattern recognition Professor Aly A. Farag Computer Vision and Image Processing Laboratory University of Louisville URL: www.cvip.uofl.edu ; E-mail: [email protected] Planned for ECE 620 and ECE 655 - Summer 2011 TA/Grader: Melih Aslan; CVIP Lab Rm 6, [email protected] Lecture 16 – Linear Discriminant Analysis Introduction • In chapter 3, the underlying probability densities were known (or given) • The training sample was used to estimate the parameters of these probability densities (ML, MAP estimations) • In this chapter, we only know the proper forms for the discriminant functions: similar to non-parametric techniques • They may not be optimal, but they are very simple to use 2 Linear discriminant functions and decisions surfaces • Definition It is a function that is a linear combination of the components of x g(x) = wtx + w0 (1) where w is the weight vector and w0 the bias • A two-category classifier with a discriminant function of the form (1) uses the following rule: Decide 1 if g(x) > 0 and 2 if g(x) < 0 Decide 1 if wtx > -w0 and 2 otherwise 3 4 – The equation g(x) = 0 defines the decision surface that separates points assigned to the category 1 from points assigned to the category 2 – When g(x) is linear, the decision surface is a hyperplane – Algebraic measure of the distance from x to the hyperplane (interesting result!) 5 6 – In conclusion, a linear discriminant function divides the feature space by a hyperplane decision surface 7 w w H) d(0, particular in w ) x ( g r therefore w w . w and 0 g(x) ce sin ) 1 w w and x x - with colinear is (since w w w . r x x 2 t p p = = = = = + = – The multi-category case • We define c linear discriminant functions and assign x to i if gi(x) > gj(x) j i ; in case of ties, the classification is undefined • In this case, the classifier is a “linear machine” • A linear machine divides the feature space into c decision regions, with gi(x) being the largest discriminant if x is in the region R i • For a two contiguous regions R i and R j ; the boundary that separates them is a portion of hyperplane Hij defined by: 8 c 1,..., i w x w ) x ( g i t i i = + = j i j i ij w w g g ) H , x ( d-- = 9 – It is easy to show that the decision regions for a linear machine are convex, this restriction limits the flexibility and accuracy of the classifier 10 Class Exercises • Ex. 13 p.159 • Ex. 3 p.201 • Write a C/C++/Java program that uses a k-nearest neighbor...
View Full Document

{[ snackBarMessage ]}

### Page1 / 29

Lecture 16(a) - Linear Discriminant Analysis - Pattern...

This preview shows document pages 1 - 12. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online