Stat841f09 - Wiki Course Notes

It assumes that inputs within each class are

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Bayes classifier was the difficulty in estimating a joint density in a multidimensional space. Naive Bayes classifiers are one possible solution to the problem. They are especially popular for problems with high- dimension feature problems. A naive Bayes classifier applies a strong independence assumption to the class density . It assumes that inputs within each class are conditionally independent. In other words, it assumes that the value of one feature in a class is unrelated to that of any other feature. Each of the d marginal densities can be estimated separately using one- dimensional density estimates. If one of the components estimated using a histogram. We can thus mix discrete and continuous variables in a naive Bayes classifier. is discrete then its density can be Naive Bayes classifiers often perform extremely well in practice despite these 'naive' and seemingly optimistic assumptions. This is because while individual class density estimates could be biased, the bias does not carry through to the posterior probabilities. It is also possible to train naive Bayes classifiers using maximum likelihood estimation. Decis ion Trees Decision trees[38] (http://en.wikipedia.org/wiki/Decision_tree) are highly intuitive learning methods that can be thought of as partitioning the feature space into a number of rectangles. Trees can be used for classification, regression, or both. Trees map features of a decision problem onto a conclusion, or label. We fit a tree model by minimizing some measure of impurity. For a single covariate , R2 = in a way that minimizes impurity. We denote by the proportion of observations in that we choose a point t on the real line that splits the real line into two sets R1 = . Extension: Decision Tree Analysis Decision Trees from Mind Tools (http://www.mindtools.com/dectree.html) usef ul link : Algorithm, Overfitting, Examples:[39] (http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo- 20/www/mlbook/ch3.pdf) ,[40] (http://robotics.stanford.edu/people/nilsson/MLDraftBook/ch6- ml.pdf) ,[41] (http://www.autonlab....
View Full Document

This document was uploaded on 03/07/2014.

Ask a homework question - tutors are online