CSEP 546 Data Mining/Machine Learning, Winter 2014:
Homework 1
Due: Monday, January 20th , beginning of class
1
Simpsons Paradox [14 points]
Imagine you and your friend, playing the slot machine in a casino. Having played on two separate machines
for a wh
Classification
Perceptron
Machine Learning CSEP546
Carlos Guestrin
University of Washington
January 21, 2014
Carlos Guestrin 2005-2014
1
THUS FAR, REGRESSION:
PREDICT A CONTINUOUS
VALUE GIVEN SOME INPUTS
Carlos Guestrin 2005-2014
2
Weather prediction revi
Kernels and
Support Vector
Machines
Machine Learning CSE446
Sham Kakade
University of Washington
November 1, 2016
2016 Sham Kakade
1
Announcements:
n
n
Project Milestones coming up
HW2
n
Youve implemented GD, SGD, etc
HW3 posted this week.
Lets get stat
Announcements:
Linear Regression
Machine Learning CSE546
Sham Kakade
University of Washington
2016 Sham Kakade
Oct 4, 2016
n
TA office hours posted on website
n
Recitation this week: Python
HW1 posted
Addcodes: decided by tomorrow. You will be
contacted
Clustering
K-means
Machine Learning CSE546
Sham Kakade
University of Washington
2016 Sham Kakade
November 15, 2016
Sham Kakade 2016
1
Announcements:
n
n
Project Milestones due date passed.
HW3 due on Monday
Itll be collaborative
n
HW2 grades posted today
http:/www.cs.washington.edu/education/courses/cse546/16au/
Whats learning?
Point Estimation
Machine Learning CSE546
Sham Kakade
University of Washington
2016 Sham Kakade
September 28, 2016
1
What is Machine Learning ?
2016 Sham Kakade
2
1
Machine Learning
Announcements:
Classification
Logistic Regression
n
HW1 due on Friday.
n
Today:
Review: sub-gradients,lasso
Logistic Regression
Machine Learning CSE546
Sham Kakade
University of Washington
October 13, 2016
Sham Kakade 2016
1
2016 Sham Kakade
2
Variable S
Announcements:
HW1 due on Friday.
Readings: please do them.
Project Proposals: please start thinking about it!
Today:
Simple Variable Selection
LASSO: Sparse Regression
Review:
Machine Learning CSE546
Sham Kakade
University of Washington
2016 Sham Kakade
CSE 546: Machine Learning
Lecture 1
The Central Limit Theorem
Instructor: Sham Kakade
1
The Central Limit Theorem
While true under more general conditions, the following is a rather simple proof of the central limit theorem. This
proof provides some insig
CSE 546: Machine Learning
Lecture 9
Online Learning & Margins
Instructor: Sham Kakade
1
Introduction
There are two common models of study:
Online Learning No assumptions about data generating process. Worst case analysis. Fundamental connections to
Game T
CSE 546: Machine Learning
Lecture 7
Gradient Descent and Stochastic Gradient Descent
Instructor: Sham Kakade
1
Gradient Descent and Stochastic Gradient Descent
Suppose we want to solve:
min G(w)
w
In many machine learning problems, we have that G(w) is of
CSE 546: Machine Learning
Lecture 1
Overview / Maximum Likelihood Estimation
Instructor: Sham Kakade
1
What is Machine Learning?
Machine learning is the study of algorithms which improve their performance with experience. The area combines
ideas from both
CSE 546: Machine Learning
Lecture 3
Risk of Ridge Regression
Instructor: Sham Kakade
0.1
Analysis
Let us rotate each Xi by V > , i.e.
Xi V > Xi
where V is the right matrix of the SVD of the n d matrix X (note this rotation does not alter the predictions o
CSE 546: Machine Learning
Lecture 3
Bias-Variance Tradeoff and Dimension-Free Regression
Instructor: Sham Kakade
1
Risk, in the well specified case
Suppose now that the linear model is correct. In particular, assume that:
Y = w> X +
where N (0, 2 ) and w
CSE 546: Machine Learning
Lecture 2
Least Squares
Instructor: Sham Kakade
1
Supervised Learning and Regression
We observe data:
T = (x1 , y1 ), . . . (xn , yn )
from some distribution. Our goal may be to predict the Y give some X. If Y is real, we may wis
Dimensionality Reduction
PCA
Machine Learning CSE4546
Sham Kakade
University of Washington
2016 Sham Kakade
November 8, 2016
Sham Kakade 2016
1
Announcements:
n
n
Project Milestones due date passed.
HW3 due on Monday
Itll be collaborative
n
HW2 grades po
Decision Trees
Machine Learning CSEP546
Carlos Guestrin
University of Washington
February 3, 2014
Carlos Guestrin 2005-2014
17
Linear separability
n
A dataset is linearly separable iff there exists a
separating hyperplane:
Exists w, such that:
n
n
w0 + i
Recommender
Systems
Machine Learning CSEP546
Carlos Guestrin
University of Washington
February 10, 2014
Personalization is transforming
our experience of the world
Information overload
Browsing is history
100 Hours a Minute
What do I care about?
-Need f
Clustering
K-means
Machine Learning CSEP546
Carlos Guestrin
University of Washington
February 18, 2014
Carlos Guestrin 2005-2014
1
Clustering images
Set of Images
Carlos Guestrin 2005-2014
[Goldberger et al.]
2
Clustering web search results
Carlos Guestri
Neural Networks
Machine Learning CSEP546
Carlos Guestrin
University of Washington
March 3, 2014
Carlos Guestrin 2005-2014
1
1
0.9
Logistic regression
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
n
P(Y|X) represented by:
n
Learning rule MLE:
Carlos Guestrin 2005-2014
0
http:/courses.cs.washington.edu/courses/csep546/14wi/
Whats learning?
Point Estimation
Machine Learning CSEP546
Carlos Guestrin
University of Washington
2005-2014 Carlos Guestrin
December 6, 2014
1
What is Machine Learning ?
2005-2014 Carlos Guestrin
2
1
Logistic Regression
Machine Learning CSEP546
Carlos Guestrin
University of Washington
January 27, 2014
Carlos Guestrin 2005-2014
1
Reading Your Brain, Simple Example
Pairwise classification accuracy: 85%
Person
[Mitchell et al.]
Animal
Carlos Guestrin 200
CSEP 546 Data Mining/Machine Learning, Winter 2014:
Homework 4
Due: Monday, March 3rd , beginning of class
1
Recommender systems for fun and prot [40 points]
For this problem, you will explore Matrix completion to predict movie ratings.
1.1
Matrix factori
CSEP 546 Data Mining/Machine Learning, Winter 2014:
Homework 3
Due: Monday, February 17th , beginning of class
1
Na Bayes[28 points]
ve
The following table contains data from an employee database. The database includes the status, department,
age range an
Regularization,
Ridge Regression
Machine Learning CSEP546
Carlos Guestrin
University of Washington
2005-2013 Carlos Guestrin
January 13, 2014
1
The regression problem
n
n
n
Instances: <xj, tj>
Learn: Mapping from x to t(x)
Hypothesis space:
Given, basis f
CSEP 546 Data Mining/Machine Learning, Winter 2014:
Homework 2
Due: Monday, February 3, beginning of class
1
Logistic regression [35 Points]
(Source: Hastie/Tibshirani/Friedman, Exercise 4.5) Suppose you are given two data points
at r , r + , which belong
CSE546 Machine Learning, Autumn 2016: Homework 1
Due: Friday, October 14th , 5pm
0
Policies
Writing: Please submit your HW as a typed pdf document (not handwritten). It is encouraged you latex
all your work, though you may use another comparable typesetti