This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Pattern recognition Professor Aly A. Farag Computer Vision and Image Processing Laboratory University of Louisville URL: www.cvip.uofl.edu ; Email: [email protected] Planned for ECE 620 and ECE 655  Summer 2011 TA/Grader: Melih Aslan; CVIP Lab Rm 6, [email protected] Lecture 16 – Linear Discriminant Analysis Introduction • In chapter 3, the underlying probability densities were known (or given) • The training sample was used to estimate the parameters of these probability densities (ML, MAP estimations) • In this chapter, we only know the proper forms for the discriminant functions: similar to nonparametric techniques • They may not be optimal, but they are very simple to use 2 Linear discriminant functions and decisions surfaces • Definition It is a function that is a linear combination of the components of x g(x) = wtx + w0 (1) where w is the weight vector and w0 the bias • A twocategory classifier with a discriminant function of the form (1) uses the following rule: Decide 1 if g(x) > 0 and 2 if g(x) < 0 Decide 1 if wtx > w0 and 2 otherwise 3 4 – The equation g(x) = 0 defines the decision surface that separates points assigned to the category 1 from points assigned to the category 2 – When g(x) is linear, the decision surface is a hyperplane – Algebraic measure of the distance from x to the hyperplane (interesting result!) 5 6 – In conclusion, a linear discriminant function divides the feature space by a hyperplane decision surface 7 w w H) d(0, particular in w ) x ( g r therefore w w . w and 0 g(x) ce sin ) 1 w w and x x  with colinear is (since w w w . r x x 2 t p p = = = = = + = – The multicategory case • We define c linear discriminant functions and assign x to i if gi(x) > gj(x) j i ; in case of ties, the classification is undefined • In this case, the classifier is a “linear machine” • A linear machine divides the feature space into c decision regions, with gi(x) being the largest discriminant if x is in the region R i • For a two contiguous regions R i and R j ; the boundary that separates them is a portion of hyperplane Hij defined by: 8 c 1,..., i w x w ) x ( g i t i i = + = j i j i ij w w g g ) H , x ( d = 9 – It is easy to show that the decision regions for a linear machine are convex, this restriction limits the flexibility and accuracy of the classifier 10 Class Exercises • Ex. 13 p.159 • Ex. 3 p.201 • Write a C/C++/Java program that uses a knearest neighbor...
View
Full Document
 Summer '08
 Staff
 Linear Algebra, Image processing, Vector Space, WI, Det, Statistical classification

Click to edit the document details