Unformatted text preview: from the line in other ranges which we have no training point. Because of that the high degree polynomail has very poor
predictive result on test cases. This is an example of overfitting model.
2. Now consider a training set which comes from a polynomial of degree two model. If we model this training set with a polynomial of degree one, our model will have
wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 42/74 10/09/2013 Stat841 - Wiki Cour se Notes high error rate on the training set, and is not complex enough to describe the problem.
3. Consider a simple classification example. If our classification rule takes as input only the colour of a fruit and concludes that it is a banana, then it is not a good
classifier. The reason is that just because a fruit is a yellow, does not mean that it is a banana. We can add complexity to our model to make it a better classifier by
considering more features typical of bananas, such as size and shape. If we continue to make our model more and more complex in order to improve our classifier,
we will eventually reach a point where the quality of our classifier no longer improves, ie., we have overfit the data. This occurs when we have considered so many
features that we have perfectly described the existing bananas, but if presented with a new banana of slightly different shape than the existing bananas, for example, it
cannot be detected. This is the tradeoff; what is the right level of complexity? Complexity Control - Nov 2, 2009
Overfitting occurs when the model becomes too complex and underfitting occurs when it is not complex enough, both of which are not desirable. To control complexity, it is
necessary to make assumptions for the model before fitting the data. Assumptions that we can make for a model are with polynomials or a neural network. There are other
ways as well.
We do not want a model to get too complex, so we control it by making an assumption on the model. With complexity control, we
want a model or a classifier with a low error rate. The lecture will explain the tradeoff betw...
View Full Document
This document was uploaded on 03/07/2014.
- Winter '13