{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

lecture10-annotated - Machine Learning 10-701/15-781 Fall...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon
1 © Eric Xing @ CMU, 2006-2008 1 Machine Learning Machine Learning 10 10- 701/15 701/15- 781, Fall 2008 781, Fall 2008 Computational Learning Theory Computational Learning Theory Eric Xing Eric Xing Lecture 10, October 8, 2008 Reading: Chap. 7 T.M book © Eric Xing @ CMU, 2006-2008 2 Generalizability of Learning z In machine learning it's really generalization error that we care about, but most learning algorithms fit their models to the training set. z Why should doing well on the training set tell us anything about generalization error? Specifically, can we relate error on to training set to generalization error? z Are there conditions under which we can actually prove that learning algorithms will work well?
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
2 © Eric Xing @ CMU, 2006-2008 3 Complexity of Learning z The complexity of leaning is measured mainly along two axis: Information Information and computation . The Information complexity is concerned with the generalization performance of learning; z How many training examples are needed? z How fast do learner’s estimate converge to the true population parameters? etc. The Computational complexity concerns the computation resources applied to the training data to extract from it learner’s predictions. It seems that when an algorithm improves with respect to one of these measures it deteriorates with respect to the other. © Eric Xing @ CMU, 2006-2008 4 What General Laws constrain Inductive Learning? These results only useful wrt O(…) ! z Sample Complexity z How many training examples are sufficient to learn target concept? z Computational Complexity z Resources required to learn target concept? z Want theory to relate: z Training examples z Quantity z Quality m z How presented z Complexity of hypothesis/concept space H z Accuracy of approx to target concept ε z Probability of successful learning δ
Background image of page 2