Stat841f09 - Wiki Course Notes

# Comwindexphptitle stat841pr intable yes 4274 10092013

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: from the line in other ranges which we have no training point. Because of that the high degree polynomail has very poor predictive result on test cases. This is an example of overfitting model. 2. Now consider a training set which comes from a polynomial of degree two model. If we model this training set with a polynomial of degree one, our model will have wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 42/74 10/09/2013 Stat841 - Wiki Cour se Notes high error rate on the training set, and is not complex enough to describe the problem. 3. Consider a simple classification example. If our classification rule takes as input only the colour of a fruit and concludes that it is a banana, then it is not a good classifier. The reason is that just because a fruit is a yellow, does not mean that it is a banana. We can add complexity to our model to make it a better classifier by considering more features typical of bananas, such as size and shape. If we continue to make our model more and more complex in order to improve our classifier, we will eventually reach a point where the quality of our classifier no longer improves, ie., we have overfit the data. This occurs when we have considered so many features that we have perfectly described the existing bananas, but if presented with a new banana of slightly different shape than the existing bananas, for example, it cannot be detected. This is the tradeoff; what is the right level of complexity? Complexity Control - Nov 2, 2009 Overfitting occurs when the model becomes too complex and underfitting occurs when it is not complex enough, both of which are not desirable. To control complexity, it is necessary to make assumptions for the model before fitting the data. Assumptions that we can make for a model are with polynomials or a neural network. There are other ways as well. We do not want a model to get too complex, so we control it by making an assumption on the model. With complexity control, we want a model or a classifier with a low error rate. The lecture will explain the tradeoff betw...
View Full Document

## This document was uploaded on 03/07/2014.

Ask a homework question - tutors are online