Unformatted text preview: ibshirani 12 ESL Chapter 1 — Introduction Trevor Hastie and Rob Tibshirani 13 ESL Chapter 1 — Introduction Trevor Hastie and Rob Tibshirani Spectral Band 1 Spectral Band 2 Spectral Band 3 Spectral Band 4 Land Usage Predicted Land Usage 14 ESL Chapter 1 — Introduction Trevor Hastie and Rob Tibshirani The Supervised Learning Problem
Starting point:
• Outcome measurement Y (also called dependent variable, response,
target)
• Vector of p predictor measurements X (also called inputs, regressors,
covariates, features, independent variables)
• In the regression problem, Y is quantitative (e.g price, blood
pressure)
• In the classiﬁcation problem, Y takes values in a ﬁnite, unordered set
(survived/died, digit 09, cancer class of tissue sample)
• We have training data (x1 , y1 ), . . . , (xN , yN ). These are observations
(examples, instances) of these measurements. 15 ESL Chapter 1 — Introduction Trevor Hastie and Rob Tibshirani Objectives
On the basis of the training data we would like to:
• Accurately predict unseen test cases
• Understand which inputs affect the outcome, and how
• Assess the quality of our predictions and inferences 16 ESL Chapter 1 — Introduction Trevor Hastie and Rob Tibshirani Philosophy
• It is important to understand the ideas behind the various techniques,
in order to know how and when to use them.
• One has to understand the simpler methods ﬁrst, in order to grasp the
more sophisticated ones.
• It is important to accurately assess the performance of a method, to
know how well or how badly it is working [simpler methods often
perform as well as fancier ones!]
• This is an exciting research area, having important applications in
science, industry and ﬁnance. 17 ESL Chapter 1 — Introduction Trevor Hastie and Rob Tibshirani Unsupervised learning
• No outcome variable, just a set of predictors (features) measured on a
set of samples.
• objective is more fuzzy — ﬁnd groups of samples that behave
similarly, ﬁnd features that behave similarly, ﬁnd linear combinations
of features with the most variation.
• difﬁcult to know how well your are doing.
• different from supervised learning, but can be useful as a
preprocessing step for supervised learning 18 ESL Chapter 1 — Introduction Trevor Hastie and Rob Tibshirani The Netﬂix prize
• competition started in October 2006. Training data is ratings for
18, 000 movies by 400, 000 Netﬂix customers, each rating between 1
and 5
• training data is very sparse— about 98% missing
• objective is to predict the rating for a set of 1 million customermovie
pairs that are missing in the training data
• Netﬂix’s original algorithm achieved a root MSE of 0.953. The ﬁrst
team to achieve a 10% improvement wins 1 million dollars.
• is this a supervised or unsupervised problem? 19 ESL Chapter 1 — Introduction Trevor Hastie and Rob Tibshirani Grand Prize: one million dollars, if beat Netﬂix’s RMSE by 10%.
Competition ends Sep 21, 2009 after ≈ 3 years, two leaders, 41305
teams! Winner is BellKor’s Pragmatic Chaos.
20...
View
Full
Document
This note was uploaded on 01/14/2014 for the course STATS 315A taught by Professor Tibshirani,r during the Winter '10 term at Stanford.
 Winter '10
 TIBSHIRANI,R
 Statistics

Click to edit the document details