10_Overfitting

10_Overfitting - 10/7/2009 Accuracy Vs. Complexity Many...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
10/7/2009 1 Overfitting Accuracy Vs. Complexity Many hypotheses. How to choose? Pareto charts Example k 1 ω 1 2 + k 2 ω 2 2 k 3 ω 1 ω 2 cos( θ 1 θ 2 )– k 4 cos( θ 1 ) – k 5 cos( θ 2 ) log error] A k 1 θ 2 k 2 ω 1 2 k 3 ω 2 2 + k 4 ω 1 ω 2 cos( θ 1 k 5 θ 2 ) + k 6 cos( θ 2 ) + k 7 cos( θ 1 ) – k 8 cos( k 9 θ 2 ) – k 10 cos( k 11 k 12 θ 2 ) - k 1 ω 1 2 k 1 ω 2 2 + k 1 ω 1 ω 2 cos( θ 2 ) + k 1 cos( θ 2 ) + k 1 cos( θ 1 ) 0 -0.4 08 More Predictive -k 1 ω 1 k 2 ω 2 + k 3 ω 1 cos( θ 1 θ 2 ) + k 4 ω 2 cos( θ 1 θ 2 ) k 1 ω 1 ω 2 k 2 cos( θ 1 θ 2 ) Parsimony [-nodes] Predictive Ability [- ω 2 · cos( θ 1 θ 2 ) + ω 1 -2 -0.8 -1.2 -1.6 Simple Complex Less Predictive Inductive Learning Setting New examples Learning as Prediction: Learner induces a general rule h from a set of observed examples that classifies new examples accurately. Assume source is stationary and samples are representative . h: X Y Testing Machine Learning Algorithms Real world Process (x 1 ,y 1 ), …, (x n ,y n ) Learner (x n+1 ,y n+1 ), … Training Data D train Test Data D test drawn randomly drawn randomly h D train Machine Learning Experiment: Gather training examples D train Run learning algorithm on D train to produce h Gather Test Examples D test Apply h to D test and measure how many test examples are predicted correctly by h Test/Training Split Real world Process (x 1 ,y 1 ), …, (x n ,y n ) Learner (x n+1 ,y n+1 ), … Training Data D train Test Data D test drawn randomly drawn randomly h D train Real world Process (x 1 ,y 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 05/30/2010 for the course CS 4700 taught by Professor Joachims during the Fall '07 term at Cornell.

Page1 / 3

10_Overfitting - 10/7/2009 Accuracy Vs. Complexity Many...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online