13-PracticalMachineLearning

Hase et althe elements of stascal learning data mining

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ts of Sta%s%cal Learning: Data Mining, Inference, and Predic%on”, Springer (2009) Wisdom of Crowds The collec%ve knowledge of a diverse and independent body of people typically exceeds the knowledge of any single individual, and can be harnessed by vo%ng. James Surowiecki h4p://socialmedia4srm.wordpress.com/ Ensemble Methods •  A single decision tree does not perform well •  But, it is super fast •  What if we learn mul%ple trees? For mul%ple trees we need even more data! Bootstrap •  •  •  •  Resampling method from sta%s%cs Useful to get error bars on es%mates Take N data points Draw N %mes with replacement •  Get es%mate from each bootstrapped sample Bagging •  Bootstrap aggregating •  Sample with replacement from your data set •  Learn a classifier for each bootstrap sample •  Average the results Bagging Example x2 x1 Bagging •  •  •  •  Reduces overfitng (variance) Normally uses one type of classifier Decision trees are popular Easy to parallelize Boos%ng •  Also ensemble method like Bagging •  But: –  weak learners evolve over %me –  votes are weighted •  Be4er than Bagging...
View Full Document

Ask a homework question - tutors are online