O oo o o ooo oo oo o o o o oo o 6 o o 4 2 o 0 2 4

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: spanned by µ∗ and lose nothing ˆk 31 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani 32 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani Can project onto even lower dimensions, using the principal components of µ∗ : ˆk • compute MK ×p , the matrix of centroids, and its sphered version ˆ M ∗ = M Σ−1/2 • Reduce M ∗ by principal components; i.e. compute B ∗ = covariance matrix of M ∗ (with mass πk on each row): B ∗ = M ∗ T Dπ M ∗ . Then compute the eigen-decomposition of B ∗ : B ∗ = V ∗ DB V ∗ T • z = v T x is the th discriminant (or canonical) variable, with ˆ v = Σ−1/2 v ∗ 33 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani 4 Linear Discriminant Analysis o oo o oo oo o oo 0 -2 -4 Coordinate 2 for Training Data 2 o o oo o o oo oo o o oo o o oo ooo oo o oo o o ooo ooo o oo o o o o o o oo o oo o o o o • oo oo oo oo o • o o ooo o o o oo o oo o oo o o o o ooo o o o o ooo ooo o o o o o o o o oo o oo o o • o oo oo o oo o oo oo o o •o o o o o oo o o o o o oo o o oo oo o oo o o o o o o o o o o o o o oo o oo oo o o o o oo o o o o o o oo o o oo oo o o oo ooo ooo o o oo oo oo oo •o o o o o ooo o o oo o o o o oo o oo o o o o o • ooo o o oo oo o o o o o o • o oo oo o o ooo o oo oo o oo o o oo o o oo o o o oo oo o o oo o oo o o o o o o o o oo o o oo o oo o o oo o oo o o oo o ooo o oo o o oo o o o o o o oo o oo o oo ooo o oo o oo oo o o o o o• oo o oo o oo o o o o o o •o o oo o o o o o o oo oo o o ooo o o o ooo o o o o o oo • o o oo o oo o o o ooo oo oo o o oo o o oo oo o o o oo o o oo o o o oo o oo oo o o o oo o •o o o o o oo oo o oo o oo o o ooo oo oo o o o o • • • • • • • • • • • oo o -6 o o -4 -2 o 0 2 4 Coordinate 1 for Training Data A two-dimensional plot of the vowel training data. There are eleven classes with X ∈ R10 , and this is the best view in terms of a LDA model. The heavy circles are the projected mean vectors for each class. The class overlap is considerable. 34 ESL Chapter 4 — Linear Methods for Classification Trevor Hastie and Rob Tibshirani Projections onto pairs of discriminant variates Linear Discriminant Analysis o o o o o • • • • -4 -2 2 • ••• •• • • •• • • •• o o o o o 0 2 4 o -6 • • • 1 2 • o o o o -2 • 0 •• • • ••• • • -1 o o oo oo o o o o o oo oo o o o o o oo oo o oo o o o o...
View Full Document

Ask a homework question - tutors are online