Homework 2
Linear Regression, Perceptron, Decision Trees and Neural Networks
CMU 10-601: Machine Learning (Fall 2015)
http:/www.cs.cmu.edu/~10601b/
OUT: Sep 23, 2015
DUE: Oct 1, 2015, 10:30 AM
START HERE: Instructions
The homework is due at 10:20 am on T

Homework 3
K-means, Hierarchical clustering, Mixture model and PCA
CMU 10-601: Machine Learning (Fall 2015)
http:/www.cs.cmu.edu/~10601b/
OUT: Oct. 1, 2015
DUE: Oct 13, 2015, 10:30 AM
START HERE: Instructions
The homework is due at 10:30 am on Tuesday Oc

10-601 Machine Learning: Homework 3 Solutions
Due 5 p.m. Wednesday, February 4, 2015
Problem 1: Implementing Naive Bayes
In this question you will implement a Naive Bayes classier for a text classication problem. You will be
given a collection of text art

Homework 2
Perceptrons, Linear Regression, Neural Networks,
Nearest Neighbors, & Decision Trees
CMU 10-601: Machine Learning (Spring 2014)
http:/www.cs.cmu.edu/~10601b/
OUT: Feb 5, 2014
DUE: Feb 19, 2014, 10:20 AM
START HERE: Instructions
The homework is

Nave Bayes Classier
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom
Mitchell, William Cohen, Eric Xing. Thanks!
Example: Live in Sq Hill? P(S|G,D,E)
S=1 iff live in Squirrel Hill
G=1 iff shop at

Clustering: Mixture Models
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom Mitchell, Ziv-
Bar Joseph, and Eric Xing. Thanks!
Problem with K-means
Hard Assignment of Samples into Three
Cl

Probability Overview
Machine Learning 10-601B
Many of these slides are derived from Tom
Mitchell, William Cohen, Eric Xing. Thanks!
Course Overview
Course website: hGp:/www.cs.cmu.edu/~10601b
Lecture notes, recitaKon not

Linear Regression
Machine Learning 10-601
Seyoung Kim
Many of these slides are derived from Tom
Mitchell. Thanks!
Regression
So far, weve been interested in learning P(Y|X) where Y has
discrete values (called classi

Semi-supervised Learning
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom
Mitchell and Ziv Bar-Joseph. Thanks!
Can Unlabeled Data improve supervised
learning?
Important quesFon! In many

10-601 Machine Learning, Fall 2009: Midterm
Andruy Tynyuk
Monday, November 2nd 2 hours
1. Personal info:
Name:
Andrew account:
E-mail address:
2. You are permitted two pages of notes and a calculator. Please turn o all cell phones and other
noisemakers

Midterm Review
Machine Learning 10-601B
Seyoung Kim
Basic on Probability
Condi9onal probability
Independence, condi9onal independence
Bayes rule, prior, likelihood, posterior probability
Chain rule
Probability Es7ma7on

Support Vector Machine II
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived fromTom
Mitchell, Ziv Bar-Joseph. Thanks!
1
Mo1va1on for Max Margin Classier
Many more possible
classiers
2
Max margi

Bayesian Networks: Inference
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from William
Cohen. Thanks!
Example: Bayesian networks for cancer
detec9on
Bayesian network: Inference
Once the netw

Hidden Markov Models I
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom
Mitchell, Ziv Bar-Joseph. Thanks!
Learning Bayes Net Structure
How can we learn Bayes Net graph structure?
In ge

Margins, Mistakes, Ranking, Sequential
Learning and the Perceptron
William Cohen
February 10, 2015
1
Online mistake-bounded learning for classication
Bayesian probability is the most-studied mathematical model of learning
(here at CMU, anyway). But there

Homework 1
Probability, Naive Bayes, and Logistic Regression
CMU 10-601: Machine Learning (Spring 2014)
http:/www.cs.cmu.edu/~10601b/
OUT: Jan 22, 2014
DUE: Feb 5, 2014, 10:20 AM
START HERE: Instructions
The homework is due at 10:20 am on Wednesday Febru

10-701 Midterm Exam, Spring 2011
1. Personal info:
Name:
Andrew account:
E-mail address:
2. There are 14 numbered pages in this exam (including this cover sheet).
3. You can use any material you brought: any book, notes, and print outs. You cannot
use

Clustering: Mixture Models
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom Mitchell, Ziv-
Bar Joseph, and Eric Xing. Thanks!
Problem with K-means
Hard Assignment of Samples into Three
Cl

Clustering: Hierarchical Clustering and K-
Means Clustering
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from William
Cohen, Ziv Bar-Joseph, Eric Xing. Thanks!
Two Classes of Learning Problems

Probabilis)c Graphical Models
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom
Mitchell, Ziv Bar-Joseph. Thanks!
Nave Bayes Classier Revisited
Full joint distribuHon:
exponenHal number of

Perceptron
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from William
Cohen. Thanks!
Perceptron
LogisCc regression is a linear classier
Another famous linear classier
The perceptron
Probabil

Support Vector Machine II
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived fromTom
Mitchell, Ziv Bar-Joseph. Thanks!
1
Max margin classiers
Instead of Hng all points, focus on boundary points

Support Vector Machine
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived fromTom
Mitchell, Ziv Bar-Joseph. Thanks!
Types of classiers
We can divide the large variety of classicaGon approaches into

Nave Bayes Classier
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom
Mitchell, William Cohen, Eric Xing. Thanks!
Lets learn classifiers by learning P(Y|X)
Consider Y=Wealth, X=<Gender, HoursWorked>

Probability Es-ma-on
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom
Mitchell, William Cohen, Eric Xing. Thanks!
Overview
Joint probability distribuIon
A funcIonal mapping f: X->Y via

Bayesian Networks
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom
Mitchell. Thanks!
Learning of Bayes Nets
Four categories of learning problems
Graph structure may be known/unknown
V

Hidden Markov Models I
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom
Mitchell, Ziv Bar-Joseph. Thanks!
Whats wrong with Bayesian networks
Bayesian networks are very useful for modelin

Boos$ng
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived fromTom Mitchell, Ziv-
Bar Joseph. Thanks!
Simple Learners
Simple (a.k.a. weak) learners are good
e.g., nave Bayes, logisIc regression, dec

Learning Theory, Over1ng, Bias Variance
Decomposi9on
Machine Learning 10-601B
Seyoung Kim
Many of these slides are derived from Tom Mitchell, 1 Ziv-
Bar Joseph. Thanks!
Any(!) learner
that outputs
a hypothesis
consistent
with

Perceptron, Margins,
Support Vector Machines
Maria-Florina Balcan
09/19/2016
Admin
HWK 3 due on Monday Sept. 26th.
Midterm on Oct 10th.
Outline for Today
Perceptron a simple learning algorithm for
supervised classification analyzed via geometric
margin