Ch4-Pattern_Classification-OLD

Ch4-Pattern_Classification-OLD - Speech Recognition Pattern...

Info iconThis preview shows pages 1–11. Sign up to view the full content.

View Full Document Right Arrow Icon
Speech Recognition Pattern Classification
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2/13/12 Veton Këpuska 2 Pattern Classification  u Introduction  u Parametric classifiers  u Semi-parametric classifiers  u Dimensionality reduction  u Significance testing 
Background image of page 2
2/13/12 Veton Këpuska 3 Pattern Classification u Goal:  To classify objects (or patterns) into categories (or  classes)  u Types of Problems:  1. Supervised Classes are known beforehand, and data samples of  each class are available  2. Unsupervised Classes (and/or number of classes) are not known  beforehand, and must be inferred from data  Feature Extraction Classifier Class i Feature Vectors x Observation s
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2/13/12 Veton Këpuska 4 Probability Basics  u Discrete probability mass function (PMF):  P ( i ω ) u Continuous probability density function (PDF):  p(x) u Expected value:  E(x)   = i i P 1 ) ( ϖ = 1 ) ( dx x p = dx x xp x E ) ( ) (
Background image of page 4
2/13/12 Veton Këpuska 5 Kullback-Liebler Distance  u Can be used to compute a distance between two probability  mass distributions,  P ( zi ), and  Q ( zi) u Makes use of inequality log  ≤  - 1  u Known as relative entropy in information theory u The divergence of  P ( zi ) and  Q ( zi)  is the symmetric sum ( 29 ( 29 ( 29 ( 29 0 log || = i i i i z Q z P z P Q P D ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 = - = - i i i i i i i i i i i z P z Q z Q z P z P z Q z P z P 0 1 log ( 29 ( 29 P Q D Q P D || || +
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2/13/12 Veton Këpuska 6 Bayes Theorem u Define: {w i} a set of M mutually exclusive classes P(w i) a priori  probability for class w i p( x |w i) PDF for feature vector  x  in class w i P(w i| x ) A posteriori probability of w i given  x
Background image of page 6
2/13/12 Veton Këpuska 7 Bayes Theorem Bayes Rule: From Bayes Rule: Where:   ) ( ) ( ) | ( ) | ( x p P x p x P i i i ϖ = = = M i i i P x p x p 1 ) ( ) | ( ) ( ) ( ) | ( ) ( ) | ( i i i P x p x p x P =
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2/13/12 Veton Këpuska 8 Bayes Decision Theory  u The probability of making an error given  is:   P(error| x )=1-P( w i| x ) if decide class w i u To minimize  P ( error | x ) (and  P ( error )):   Choose w i if  P(w i| x )>P(w j| x )   j ≠i
Background image of page 8
2/13/12 Veton Këpuska 9 Bayes Decision Theory u For a two class problem this decision rule means:  Choose w 1  if else  w 2  u This rule can be expressed as a likelihood ratio:  ) ( ) ( ) | ( ) ( ) ( ) | ( 2 2 1 1 x p P x p x p P x p ϖ ) ( ) ( ) | ( ) | ( 1 2 2 1 P P x p x p
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2/13/12 Veton Këpuska 10 Bayes Risk  u Define cost function  ij  λ and conditional risk  R ( i ω | x ):  n ij  λ is cost of classifying  as  ω when it is really  ω n R ( i ω | x ) is the risk for classifying  as class  i ω u Bayes risk  is the minimum risk which can be achieved:
Background image of page 10
Image of page 11
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 43

Ch4-Pattern_Classification-OLD - Speech Recognition Pattern...

This preview shows document pages 1 - 11. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online