Maths Refresher
Piyush Rai
Machine Learning (CS771A)
Aug 21, 2016
Machine Learning (CS771A)
Maths Refresher
1
Topics
Basics of Linear Algebra
Basics of Optimization and Convex Optimization
Basics of Probability and Probability Distributions
Some slides co

Machine Learning (CS771A)
Homework 3 (Due date: Nov 2, 2016, 11:59pm)
Instructions
Each late submission will receive a 10% penalty per day for up to 2 days. No submissions will be accepted
after the 2nd late day.
We will only accept electronic submissio

Machine Learning (CS771A)
Homework 2 (Due date: Oct 5, 2016, 11:59pm)
Instructions
Each late submission will receive a 10% penalty per day for up to 5 days. No submissions will be accepted
after the 5th late day.
We will only accept electronic submissio

Review of Probability Theory
Arian Maleki and Tom Do
Stanford University
Probability theory is the study of uncertainty. Through this class, we will be relying on concepts
from probability theory for deriving machine learning algorithms. These notes attem

Machine Learning and Software Engineering
in Health Informatics
David A. Clifton , Jeremy Gibbons , Jim Davies , Lionel Tarassenko
Institute
of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
Department of Com

Chapter 9: Correlation and Regression: Solutions
9.1 Correlation
In this section, we aim to answer the question: Is there a relationship between A and B?
Is there a relationship between the number of employee training hours and the number of
on-the-job ac

Probabilistic Models for Classification:
Logistic Regression
Piyush Rai
Machine Learning (CS771A)
Aug 17, 2016
Machine Learning (CS771A)
Probabilistic Models for Classification: Logistic Regression
1
Logistic Regression: The Model
A model for doing probab

Learning as Optimization:
Linear Regression
Piyush Rai
Machine Learning (CS771A)
Aug 10, 2016
Machine Learning (CS771A)
Learning as Optimization: Linear Regression
1
Learning as Optimization
Consider a supervised learning problem with training data cfw_(x

Learning via Probabilistic Modeling of Data
Piyush Rai
Machine Learning (CS771A)
Aug 12, 2016
Machine Learning (CS771A)
Learning via Probabilistic Modeling of Data
1
Probabilistic Modeling of Data
Assume the data y = cfw_y1 , y2 , . . . , yN as generated

Learning Maximum-Margin Hyperplanes:
Support Vector Machines
Piyush Rai
Machine Learning (CS771A)
Aug 24, 2016
Machine Learning (CS771A)
Learning Maximum-Margin Hyperplanes: Support Vector Machines
1
Perceptron and (Lack of) Margins
Perceptron learns a hy

Nonlinear Learning with Kernels
Piyush Rai
Machine Learning (CS771A)
Aug 26, 2016
Machine Learning (CS771A)
Nonlinear Learning with Kernels
1
Linear Models
Linear models (e.g., linear regression, linear SVM) are nice and interpretable
but have limitations

Bayesian Learning
Note: The core of the material presented here has
been borrowed from the slides prepared by Pedro
Domingos. Minor customization has been done to
suit the specific needs of the course.

Decision Trees
Note: The core of the material presented here has been
borrowed from the slides prepared by Pedro Domingos.
Minor customization has been done to suit the specific
needs of the course.
20
10
x1
12
8
8
2
Summary: Decision Trees
Representation

Artificial Neural Networks
Note: The core of the material presented here has been
borrowed from the slides prepared by Pedro Domingos.
Minor customization has been done to suit the specific
needs of the course.

Learning via Probabilistic Modeling of Data
Piyush Rai
Machine Learning (CS771A)
Aug 12, 2016
Machine Learning (CS771A)
Learning via Probabilistic Modeling of Data
1
Quick Recap of Last Lecture
Machine Learning (CS771A)
Learning via Probabilistic Modeling

Clustering: K -means and Kernel K -means
Piyush Rai
Machine Learning (CS771A)
Aug 31, 2016
Machine Learning (CS771A)
Clustering: K -means and Kernel K -means
1
Clustering
Usually an unsupervised learning problem
Given: N unlabeled examples cfw_x 1 , . . .

Learning by Asking Questions:
Decision Trees
Piyush Rai
Machine Learning (CS771A)
Aug 5, 2016
Machine Learning (CS771A)
Learning by Asking Questions: Decision Trees
1
A Classification Problem
Indoor or Outdoor ?
Pic credit: Decision Forests: A Unified Fra

Generative Models for Dimensionality Reduction:
Probabilistic PCA and Factor Analysis
Piyush Rai
Machine Learning (CS771A)
Oct 5, 2016
Machine Learning (CS771A)
Generative Models for Dimensionality Reduction: Probabilistic PCA and Factor Analysis
1
Genera

Linear Dimensionality Reduction:
Principal Component Analysis
Piyush Rai
Machine Learning (CS771A)
Sept 2, 2016
Machine Learning (CS771A)
Linear Dimensionality Reduction: Principal Component Analysis
1
Dimensionality Reduction
Machine Learning (CS771A)
Li

Nonlinear Learning with Kernels
Piyush Rai
Machine Learning (CS771A)
Aug 26, 2016
Machine Learning (CS771A)
Nonlinear Learning with Kernels
1
Linear Models
Linear models (e.g., linear regression, linear SVM) are nice and interpretable
but have limitations

Online Learning via Stochastic Optimization,
and Perceptron
Piyush Rai
Machine Learning (CS771A)
Aug 20, 2016
Machine Learning (CS771A)
Online Learning via Stochastic Optimization, and Perceptron
1
Stochastic Optimization for Logistic Regression
Recall th

Learning as Optimization:
Linear Regression
Piyush Rai
Machine Learning (CS771A)
Aug 10, 2016
Machine Learning (CS771A)
Learning as Optimization: Linear Regression
1
Learning as Optimization
Consider a supervised learning problem with training data cfw_(

Machine Learning (CS771A)
Homework 1 (Due date: Aug 31, 2016, 11:59pm)
Instructions
Each late submission will receive a 10% penalty per day for up to 5 days. No submissions will be accepted
after the 5th late day.
We will only accept electronic submissi

Generative Models for Clustering, GMM, and
Intro to EM
Piyush Rai
Machine Learning (CS771A)
Sept 26, 2016
Machine Learning (CS771A)
Generative Models for Clustering, GMM, and Intro to EM
1
Generative Models
A probabilistic way to think about the data gene

Learning by Computing Distances:
Distance-based Methods and Nearest Neighbors
Piyush Rai
Machine Learning (CS771A)
Aug 3, 2016
Machine Learning (CS771A)
Learning by Computing Distances: Distance-based Methods and Nearest Neighbors
1
Data and Data Represen

Logistic Regression
Piyush Rai
Machine Learning (CS771A)
Aug 17, 2016
Machine Learning (CS771A)
Logistic Regression
1
Recap
Machine Learning (CS771A)
Logistic Regression
2
Linear Regression: The Optimization View
w > x n )2 and solve the following
Define

Matrix Factorization and Matrix Completion
Piyush Rai
Machine Learning (CS771A)
Sept 21, 2016
Machine Learning (CS771A)
Matrix Factorization and Matrix Completion
1
Matrix Factorization
Given a matrix X of size N M, approximate it as a product of two matr