Naive Bayes Classifiers_4

Naive Bayes Classifiers_4 - Naive Bayes Classifiers Tommy...

Info iconThis preview shows pages 1–9. Sign up to view the full content.

View Full Document Right Arrow Icon
Naive Bayes Classifiers Tommy W. S. Chow Dept. of Electronic Engineering
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Classification Methods Supervised learning of a document-label assignment function Many systems partly rely on machine learning (Google, MSN, Yahoo!, …) Naive Bayes (simple, common method) k-Nearest Neighbors (simple, powerful) Support-vector machines (new, more powerful) … plus many other methods Note that many commercial systems use a mixture of methods
Background image of page 2
Introduction Two types of classifiers Generative build a generative statistical model, i.e., LDA, Fisher LDA e.g., Bayesian classifiers Discriminative directly estimate a decision rule/boundary e.g., neural network (we’ll work on it) , decision tree
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Introduction-Naive Bayes Classifier It is a simple probabilistic classifier based on applying Bayes' theorem (from Bayesian statistics) with strong (naive) independence assumptions. In simple terms, a naive Bayes classifier assumes that the presence (or absence) of a particular feature of a class is unrelated to the presence (or absence) of any other feature, i.e., document, the appearance of “computer” is unrelated to “science” “tennis” is unrelated to “backhand” etc.
Background image of page 4
Does a patient have cancer or not? A patient takes a lab test and the result comes back positive. It is known that the test returns a correct positive result in only 99% of the cases and a correct negative result in only 95% of the cases. Furthermore, only 0.03 of the entire population has this disease. How likely that this patient has cancer? Bayesian probability.
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Bayesian Methods (generative classifier) Our focus Learning and classification methods based on probability theory. Bayes theorem plays a critical role in probabilistic learning and classification. Uses prior probability of each category given no information about an item. Categorization produces a posterior probability distribution over the possible categories given a description of an item.
Background image of page 6
Introduction Depending on the precise nature of the probability model, naive Bayes classifiers can be trained very efficiently in a supervised learning setting. In many practical applications, parameter estimation for naive Bayes models uses the method of maximum likelihood; In spite of the simplified assumptions, naive Bayes classifiers often work much better in many complex real-world situations than one might expect. In some fields, the performance of naive Bayes Classifier is better than neural networks and decision tree.
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Probability In our day to day life, the statement "there is 60% chance of rain today". This statement infers that the chance of rain is more than that having a dry weather. We decide on our breakfast from a statement that "corn flakes might reduce cholesterol".
Background image of page 8
Image of page 9
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 41

Naive Bayes Classifiers_4 - Naive Bayes Classifiers Tommy...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online