CSCD11 Machine Learning and Data Mining, Fall 2010 Assignment 3: Classication and Bayesian Methods
Due Thursday, November 18, 3pm (before tutorial) Note: This assignment comprises two theoretical ques
CSCD11 Machine Learning and Data Mining, Fall 2010 Assignment 2: Classication
Due Wednesday, Oct. 27, 3pm Note: This assignment comprises two theoretical questions and one programming question. For th
CSCD11 Machine Learning and Data Mining, Fall 2010 Assignment 1: Least-Squares Regression
Due Friday, October 1, 5pm
Note: For this assignment you will write two functions and one script in Matlab. Yo
CSC 411 / CSC D11
Introduction to Machine Learning
1 Introduction to Machine Learning
Machine learning is a set of tools that, broadly speaking, allow us to teach computers how to perform tasks by pro
CSC 411 / CSC D11
Nonlinear Regression
3 Nonlinear Regression
Sometimes linear models are not sufcient to capture the real-world phenomena, and thus nonlinear models are necessary. In regression, all
CSC 411 / CSC D11
Linear Regression
2 Linear Regression
In regression, our goal is to learn a mapping from one real-valued space to another. Linear regression is the simplest form of regression: it is
CSC 411 / CSC D11
Quadratics
4 Quadratics
The objective functions used in linear least-squares and regularized least-squares are multidimensional quadratics. We now analyze multidimensional quadratics
CSC 411 / CSC D11
Basic Probability Theory
5 Basic Probability Theory
Probability theory addresses the following fundamental question: how do we reason? Reasoning is central to many areas of human end
CSC 411 / CSC D11
Probability Density Functions (PDFs)
6 Probability Density Functions (PDFs)
In many cases, we wish to handle data that can be represented as a real-valued random variable, or a real-
CSC 411 / CSC D11
Estimation
7 Estimation
We now consider the problem of determining unknown parameters of the world based on measurements. The general problem is one of inference, which describes the
CSC 411 / CSC D11
Classication
8 Classication
In classication, we are trying to learn a map from an input space to some nite output space. In the simplest case we simply detect whether or not the inpu
CSC 411 / CSC D11
Gradient Descent
9 Gradient Descent
There are many situations in which we wish to minimize an objective function with respect to a parameter vector: w = arg min E (w) (1)
w
but no cl
CSC 411 / CSC D11
Cross Validation
10
Cross Validation
Suppose we must choose between two possible ways to t some data. How do we choose between them? Simply measuring how well they t they data would
CSC 411 / CSC D11
Bayesian Methods
11
Bayesian Methods
So far, we have considered statistical methods which select a single best model given the data. This approach can have problems, such as over-tti