This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Link between LDA and OLS Jean-Philippe Vert June 9, 2011 This is the solution to exercise 4.2 of  which shows a link between linear discriminant analysis (LDA) and ordinary least squares (OLS) in the binary case. We have features x R p and a two-class response, with class sizes N 1 , N 2 . The training patterns are denoted x 1 , . . . , x N R p , stored in the n p matrix X . We encode the class of each training point in the real number y i =- N/N 1 for patterns x i in class 1, and y i = N/N 2 for patterns x i in class 2. (a) From equation (4.11) in  we know that, in the binary case, the LDA rule classifies a pattern x to class 2 if x > - 1 ( 2- 1 ) > 1 2 > 2 - 1 2- 1 2 > 1 - 1 1 + log N 1 N- log N 2 N , (1) and class 1 otherwise. (b) Let us introduce a few more notations. Let U i R n be the class indicator vector of class i , and U = U 1 + U 2 be the vector with all entries equal to 1. When we encode class 1 (resp. class 2) by the real number a 1 =- N/N 1 (resp. a 2 = N/N 2), the vector of labels becomes Y = a 1 U 1 + a 2 U 2 ....
View Full Document