CS340 Machine learning
Nave Bayes classifiers
Document classification
Let Y cfw_1,C be the class label and x cfw_0,1d
eg Y cfw_spam, urgent, normal,
xi = I(word i is present in message)
Bag of word
CPSC 340 Assignment 6 (due December 4)
Multi-Class Logistic, Label Propagation with Random Walks
You can work in groups on the assignments. However, please hand in your own assignments and state
the
CPSC 340 Assignment 1 (due September 18)
Summary Statistics and Data Visualization, Decision Tress and Cross-Validation, Probability
You can work in groups on the assignments. However, please hand in
CPSC 340 Assignment 2 (due October 2nd)
Frequency-Based Supervised Learning, K-Means Clustering
You can work in groups on the assignments. However, please hand in your own assignments and state
the g
CPSC 340 Assignment 4 (due November 13)
Regularized Logistic Regression, PCA, Outlier Detection
You can work in groups on the assignments. However, please hand in your own assignments and state
the g
CPSC 340 Assignment 3 (due October 23rd)
Clustering, Item Recommendation, Linear Regression
You can work in groups on the assignments. However, please hand in your own assignments and state
the group
CPSC 340 Assignment 5 (due November 27)
Sparse Latent Factors, Recommender Systems, MDS, Neural Networks
You can work in groups on the assignments. However, please hand in your own assignments and st
CPSC 340 Assignment 1 (due September 23)
Data Exploration, Decision Trees, Training and Testing, Naive Bayes
You can work on your own or in a group of 2. If you work in a group, please only hand in o
CPSC 340 Assignment 2 (due October 7)
K-Nearest Neighbours, Random Forests, K-Means, Density-Based Clustering
1
K-Nearest Neighbours
In this question we revisit the citiesSmall dataset from the previo
Steinborn Homes manufactures prefabricated chalets in Colorado. The company uses a perpetual
inventory system and a job cost system in which each chalet is a job. The following events
occurred during
CS340 Machine learning
Midterm review
Topics
Bayesian statistics
Information theory
Decision theory
kNN not on exam
Sampling distributions (confidence intervals etc) not on exam
Bayesian belief upd
CS340 Machine learning
Bayesian model selection
Bayesian model selection
Suppose we have several models, each with
potentially different numbers of parameters.
Example: M0 = constant, M1 = straight
CS340 Machine learning
Lecture 2
Classification and generalization error
Summary of last lecture
Given training data D = cfw_ (x1.y1), , (xN, yN)
Choose right hypothesis class H
linear
quadratic
De
CS340 Machine learning
Lecture 4
K-nearest neighbors
Nearest neighbor classifier
Remember all the training data (non-parametric
classifier)
At test time, find closest example in training set,
and re
CS340 Machine learning
Information theory
1
Announcements
If you did not get email, contact [email protected]
Newsgroup ubc.courses.cpsc.340
Hw1 due wed bring hardcopy to start of class
Added knnClassi
CS340 Machine learning
Lecture 5
Notes
Outline
HW1
Finish KNN
Start info theory
Office hours Tue 4-5, CS187
Standard error
Suppose we want to estimate E[X] from n samples,
X1, , Xn (eg X is genera
CS340 Machine learning
Decision theory
1
From beliefs to actions
We have briefly discussed ways to compute p(y|x),
where y represents the unknown state of nature (eg.
does the patient have lung cance
CS340:
Bayesian concept learning
Kevin Murphy
Based on Josh Tenenbaums PhD
thesis (MIT BCS 1999)
Concept learning (binary
classification) from positive and
negative examples
Concept learning from posi
CS340
Bayesian concept learning cont'd
Kevin Murphy
Prior p(h)
X=cfw_60,80,10,30
Why prefer multiples of 10 over even
numbers?
Size principle (likelihood)
Why prefer multiples of 10 over
multiples
CS340 Machine learning
Bayesian statistics 1
Fundamental principle of Bayesian statistics
In Bayesian stats, everything that is uncertain (e.g.,
) is modeled with a probability distribution.
We inco
CS340 Machine learning
Bayesian statistics 2
1
Binomial distribution (count data)
X ~ Binom(, N), X cfw_0,1,N
N
x
P (X = x|, N ) =
theta=0.500
x (1 )N x
theta=0.250
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
CS340 Machine learning
Bayesian statistics 3
1
Outline
Conjugate analysis of and 2
Bayesian model selection
Summarizing the posterior
2
Unknown mean and precision
The likelihood function is
p(D|,