{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

SPR_LectureHandouts_Chapter_03_Part3

SPR_LectureHandouts_Chapter_03_Part3 - Pattern Recognition...

Info icon This preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Electrical and Computer Engineering Department Saurabh Prasad Pattern Recognition Chapter 3 Pattern Recognition ECE-8443 Chapter 3, Part 3 Parameter estimation of the feature space – Practical aspects Electrical and Computer Engineering Department, Mississippi State University.
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2 Electrical and Computer Engineering Department Saurabh Prasad Pattern Recognition Chapter 3 Outline When do ML and Bayes estimation methods differ? Different sources of error Problems introduced by high dimensional feature spaces Practical aspects of ML estimates when building discriminant functions Handling inverses and determinants of covariance estimates when they are not full ranked
Image of page 2
3 Electrical and Computer Engineering Department Saurabh Prasad Pattern Recognition Chapter 3 For infinite amounts of data, the solutions converge. However, limited data is always a problem. If prior information is reliable, a Bayesian estimate can be superior. Bayesian estimates for uniform priors are similar to an ML solution. If p( θ | D) is broad or asymmetric around the true value, the approaches are likely to produce different solutions. Maximum-likelihood – versus Bayesian Estimation
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
4 Electrical and Computer Engineering Department Saurabh Prasad Pattern Recognition Chapter 3 When designing a classifier using these techniques, there are three sources of error: Bayes Error (Irreducible/Indistinguishability error): the error due to overlapping distributions Inherent property of the problem for the given features Can never be eliminated Model Error: the error due to an incorrect model or incorrect assumption about the parametric form. Estimation Error: the error arising from the fact that the parameters are estimated from a finite amount of data (unreliable statistical estimates, unstable inverses etc.) Maximum-likelihood – versus Bayesian Estimation
Image of page 4
5 Electrical and Computer Engineering Department Saurabh Prasad Pattern Recognition Chapter 3 In the limit of infinite training data, estimation error vanishes Total error will be same for both ML and Bayesian estimation approaches ML classifiers (where likelihood functions for each class are represented by parameters that are estimated by ML techniques) are simpler Lead to classifiers nearly as accurate as those based on Bayesian estimation of the parameters Maximum-likelihood – versus Bayesian Estimation
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon