10/28/2014
1.7. Naive Bayes scikit-learn 0.15.2 documentation
Home
Installation
Documentation
Examples
1.7. Naive Bayes
Naive Bayes methods are a set of supervised learning algorithms based on applying Bayes theorem
with the naive assumption of independen
Lectures /
Classification
On this page (hide)
Classification
Generative vs. Discriminative Approaches
Generative approach
Discriminative approach
Classification
The goal of classification is to learn to predict a categorical outcome from input: for exampl
Lectures /
Bias Variance
See also the unusually clear wikipedia article.
This nice applet for polynomial regression is a lot of fun (as far as regression applets go). It illustrates the fundamental trade-off in learning predictive models: bias
versus vari
Home
Installation
Documentation
Examples
1.13. Linear and quadratic discriminant analysis
Linear discriminant analysis (lda.LDA) and quadratic discriminant analysis (qda.QDA) are two classic
classifiers, with, as their names suggest, a linear and a quadra
An introduction to regression
Note to other teachers and users of
these slides. Andrew would be delighted
if you found this source material useful in
giving your own lectures. Feel free to use
these slides verbatim, or to modify them
to fit your own needs
Deep Learning for NLP
(without Magic)
Richard Socher and Christopher Manning
Stanford University
NAACL 2013, Atlanta
h0p:/nlp.stanford.edu/courses/NAACL2013/
*with a big thank you t
Linear Methods for Classication
Linear Methods for Classication
Jia Li
Department of Statistics The Pennsylvania State University
Email: jiali@stat.psu.edu http:/www.stat.psu.edu/jiali
Jia Li
http:/www.stat.psu.edu/jiali
Linear Methods for Classication
Cl
Logistic Regression
What is the logistic curve? What is the base of the natural logarithm? Why do statisticians prefer
logistic regression to ordinary linear regression when the DV is binary? How are probabilities, odds
and logits related? What is an odds
Logistic Regression
Logistic Regression
Jia Li
Department of Statistics The Pennsylvania State University
Email: jiali@stat.psu.edu http:/www.stat.psu.edu/jiali
Jia Li
http:/www.stat.psu.edu/jiali
Logistic Regression
Logistic Regression
Preserve linear cl
Lectures /
Regression
On this page (hide)
Continuous variable prediction
Linear regression
Multivariate case
What happened to the constant term?
What about polynomial or non-linear regression?
MLE
MAP
Continuous variable prediction
Suppose now instead of
Lectures /
SV Ms
On this page (hide)
Support Vector Machines
Hyperplanes and Margins
The Dual View
Kernels
Non-separable case
Solving SVMs
Support Vector Machines
Historically, Support Vector Machines (SVMs) were motivated by the notion of the maximum mar
Lectures /
Probability Review
On this page (hide)
1. Introductory Example
2. Combining Events
3. Random Variables
4. Probability Distributions
5. Joint Distributions
6. Marginalization
7. PDFs and CDFs
8. Expectation and Variance
9. Conditional Probabilit
6.034 Recitation October 23: Nearest Neighbors, Drawing decision boundaries
Bob Berwick
Boundary lines are formed by the intersection of perpendicular bisectors of every pair of points.
Using pairs of closest points in different classes gives a good enoug
Recitations /
Bias Variance
On this page (hide)
1. Bias-Variance Decomposition in Regression
1. Bias-Variance Decomposition in Regression
Suppose we want to fit the following regression prediction model:
, which is constant for all , to some data. Now, su
Lectures /
Point Estimation
On this page (hide)
Point Estimation Basics: Biased Coin
Maximum Likelihood Estimation (MLE) for the coin
Learning guarantees
The Bayesian way
Point Estimation Basics: Biased Coin
Before we get to any complex learning algorithm
Lectures /
EM
On this page (hide)
Mixtures of Gaussians and Expectation Maximization
Gaussian Mixtures
Expectation maximization
EM in general
Mixtures of Gaussians and Expectation Maximization
The K-means algorithm turns out to be a special case of cluste
Lectures /
Local Learning
On this page (hide)
The Supervised Learning Problem
Local Learning
Nearest Neighbor Methods
Non-parametric models
1-NN Consistency, Bias vs Variance
K-Nearest Neighbors
Kernel Regression
Sample code
Summary of Local Learning Meth
Lectures /
Logistic
On this page (hide)
Logistic Regression
1-D Example
2-D Example
Computing MLE
Computing MAP
Multinomial Logistic Regression
Naive Bayes vs. Logistic Regression
Linear boundary for 2-class Gaussian Naive Bayes with shared variances
Logi
Lectures /
Naive Bayes
On this page (hide)
Naive Bayes Model
Estimating
Estimating
MLE vs. MAP
Examples
Naive Bayes Model
The Naive Bayes classifier is an example of the generative approach: we will model
Distance(miles)
1 mile
2 miles
1 mile
1 mile
2 mil
Lectures /
Gaussians
On this page (hide)
Modeling continuous variables
The Gaussian Model
MLE for Gaussians
The Bayesian way
Modeling continuous variables
The biased coin example involved discrete (Bernoulli) variables, and there was not much choice in mo