Lecture 16(a) - Linear Discriminant Analysis

Lecture 16(a) - Linear Discriminant Analysis - Pattern...

Info iconThis preview shows pages 1–12. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Pattern recognition Professor Aly A. Farag Computer Vision and Image Processing Laboratory University of Louisville URL: www.cvip.uofl.edu ; E-mail: aly.farag@louisville.edu Planned for ECE 620 and ECE 655 - Summer 2011 TA/Grader: Melih Aslan; CVIP Lab Rm 6, msaslan01@lousiville.edu Lecture 16 Linear Discriminant Analysis Introduction In chapter 3, the underlying probability densities were known (or given) The training sample was used to estimate the parameters of these probability densities (ML, MAP estimations) In this chapter, we only know the proper forms for the discriminant functions: similar to non-parametric techniques They may not be optimal, but they are very simple to use 2 Linear discriminant functions and decisions surfaces Definition It is a function that is a linear combination of the components of x g(x) = wtx + w0 (1) where w is the weight vector and w0 the bias A two-category classifier with a discriminant function of the form (1) uses the following rule: Decide 1 if g(x) > 0 and 2 if g(x) < 0 Decide 1 if wtx > -w0 and 2 otherwise 3 4 The equation g(x) = 0 defines the decision surface that separates points assigned to the category 1 from points assigned to the category 2 When g(x) is linear, the decision surface is a hyperplane Algebraic measure of the distance from x to the hyperplane (interesting result!) 5 6 In conclusion, a linear discriminant function divides the feature space by a hyperplane decision surface 7 w w H) d(0, particular in w ) x ( g r therefore w w . w and 0 g(x) ce sin ) 1 w w and x x - with colinear is (since w w w . r x x 2 t p p = = = = = + = The multi-category case We define c linear discriminant functions and assign x to i if gi(x) > gj(x) j i ; in case of ties, the classification is undefined In this case, the classifier is a linear machine A linear machine divides the feature space into c decision regions, with gi(x) being the largest discriminant if x is in the region R i For a two contiguous regions R i and R j ; the boundary that separates them is a portion of hyperplane Hij defined by: 8 c 1,..., i w x w ) x ( g i t i i = + = j i j i ij w w g g ) H , x ( d-- = 9 It is easy to show that the decision regions for a linear machine are convex, this restriction limits the flexibility and accuracy of the classifier 10 Class Exercises Ex. 13 p.159 Ex. 3 p.201 Write a C/C++/Java program that uses a k-nearest neighbor...
View Full Document

Page1 / 29

Lecture 16(a) - Linear Discriminant Analysis - Pattern...

This preview shows document pages 1 - 12. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online