CSCD11 Machine Learning and Data Mining, Fall 2010 Assignment 3: Classication and Bayesian Methods
Due Thursday, November 18, 3pm (before tutorial) Note: This assignment comprises two theoretical questions and one programming question. For the theoretical
CSCD11 Machine Learning and Data Mining, Fall 2010 Assignment 2: Classication
Due Wednesday, Oct. 27, 3pm Note: This assignment comprises two theoretical questions and one programming question. For the theoretical questions hand-written or computer format
CSCD11 Machine Learning and Data Mining, Fall 2010 Assignment 1: Least-Squares Regression
Due Friday, October 1, 5pm
Note: For this assignment you will write two functions and one script in Matlab. You will hand in one tar-le containing these three les. A
CSC 411 / CSC D11
Introduction to Machine Learning
1 Introduction to Machine Learning
Machine learning is a set of tools that, broadly speaking, allow us to teach computers how to perform tasks by providing examples of how they should be done. For example
CSC 411 / CSC D11
Nonlinear Regression
3 Nonlinear Regression
Sometimes linear models are not sufcient to capture the real-world phenomena, and thus nonlinear models are necessary. In regression, all such models will have the same basic form, i.e., y = f
CSC 411 / CSC D11
Linear Regression
2 Linear Regression
In regression, our goal is to learn a mapping from one real-valued space to another. Linear regression is the simplest form of regression: it is easy to understand, often quite effective, and very ef
CSC 411 / CSC D11
Quadratics
4 Quadratics
The objective functions used in linear least-squares and regularized least-squares are multidimensional quadratics. We now analyze multidimensional quadratics further. We will see many more uses of quadratics furt
CSC 411 / CSC D11
Basic Probability Theory
5 Basic Probability Theory
Probability theory addresses the following fundamental question: how do we reason? Reasoning is central to many areas of human endeavor, including philosophy (what is the best way to ma
CSC 411 / CSC D11
Probability Density Functions (PDFs)
6 Probability Density Functions (PDFs)
In many cases, we wish to handle data that can be represented as a real-valued random variable, or a real-valued vector x = [x1 , x2 , ., xn ]T . Most of the int
CSC 411 / CSC D11
Estimation
7 Estimation
We now consider the problem of determining unknown parameters of the world based on measurements. The general problem is one of inference, which describes the probabilities of these unknown parameters. Given a mod
CSC 411 / CSC D11
Classication
8 Classication
In classication, we are trying to learn a map from an input space to some nite output space. In the simplest case we simply detect whether or not the input has some property or not. For example, we might want
CSC 411 / CSC D11
Gradient Descent
9 Gradient Descent
There are many situations in which we wish to minimize an objective function with respect to a parameter vector: w = arg min E (w) (1)
w
but no closed-form solution for the minimum exists. In machine l
CSC 411 / CSC D11
Cross Validation
10
Cross Validation
Suppose we must choose between two possible ways to t some data. How do we choose between them? Simply measuring how well they t they data would mean that we always try to t the data as closely as pos
CSC 411 / CSC D11
Bayesian Methods
11
Bayesian Methods
So far, we have considered statistical methods which select a single best model given the data. This approach can have problems, such as over-tting when there is not enough data to fully constrain the