# 3 2 1 2 22 1 2 22222 2 2 22 2 1 33 33 3 33 22 2 22 12

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 2 2 22 22 22 2 22 3 3 3 33 2 2 2 222 2 22 22 2 1 3 33333 3 22 2 2 22 2 3 33 3 3 3 3 3 2 2 22 2 2 2 2 3 3 33 2 22 2 2 2 22 2 2 2 2 3 2 3 2 2 22 2 2 22 2 222 22 2 13 3 2 2 2 22 2 2 2 2 2 22 3 333 3 3 2 1 2 22 1 2 22222 2 2 22 2 1 33 33 3 33 22 2 22 12 1 2 2 1 12 1 3 33 3 2 2 22 22 1 1 1 2 1 2 1 22 1 1 1 33 33 3 22 1 2 2 2 1 22 1 1 1 33 3 3 3 2 11 1 11 2 11 1 1 1 21 1 1 3 33 3 33 2 1 2 21 1 2 3 1 11 1 2 2 1 1 1 11 1 11 1 1 11 1 1 1 33 3 333 12 22 21 31333 3 11 1 1 3 33 1 11 2 33 3 2 111 1 1 2 1 1 33333 33 1 11 11 1 33 3 1 33 33 3 1 1 1 11 1 1 1 1 1 11 3 1 3 33 3 1 11 1 11 1 1 11 1 11 11 11 1 1 11 11 1 111 33 133 3 1 33 33 3 33 3 11 1 11 1 11 33 1 1 1 1 11 1 1 1 1 1 33 3333 3 3 1333 1 1 11 1 1 1 33 3 33 11 1 1 1 1 11 1 1 1 1 33 1 11 1 11 1 33 3 3 3333 11 3 3 3333 3 33 3 1 111 1 1 1 3 33 111 33 33 1 11 333 3 3 33 2 Two methods for ﬁtting quadratic boundaries. [Left] Quadratic decision boundaries, obtained using LDA in the ﬁve-dimensional “quadratic” space. [Right] Quadratic decision boundaries found by QDA. The differences are small, as is usually the case. 12 ESL Chapter 4 — Linear Methods for Classiﬁcation Trevor Hastie and Rob Tibshirani Regularized discriminant analysis ˆ ˆ ˆ • Regularized QDA Σk (α) = αΣk + (1 − α)Σ ˆ ˆ • Regularized LDA Σ(γ ) = γ Σ + (1 − γ )ˆ 2 I σ ˆ • Together → Σ(α, γ ) ˆ ˆ ˆ • could use Σ(γ ) = γ Σ + (1 − γ )diag(Σ) • in “Nearest Shrunken Centroid” work we use p δk (x) = j =1 (xj − µj k )2 ˆ 1 − log πk s2 2 j where µj k is a shrunken centroid. Details later. ˆ Vowel data (ESL p443 ): p = 10 features (derived from spectra); 11 classes (vowel sounds); 528 training obs, 462 test obs. 13 ESL Chapter 4 — Linear Methods for Classiﬁcation Trevor Hastie and Rob Tibshirani Regularized Discriminant Analysis on the Vowel Data • ••••••• •••••••••••••••••••••••••••• • •• •• •• •• Test Data Train Data ••••• 0.1 0.2 0.3 0.4 •••••••••• •• •• ••• •• ••••• ••••• 0.0 Misclassification Rate 0.5 •••• 0.0 0.2 0.4 0.6 •••••• ••••••••••• 0.8 1.0 α Test and training errors for the vowel data, using regularized discriminant analysis with a series of values of α ∈ [0, 1]. The optimum for the test data occurs around α = 0.9, close to quadratic discriminant analysis. 14 ESL Chapte...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online