homework1b solutions

The training error and test error are depicted below

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 15.png train-002824.png train-004496.png train-004612.png train-006031.png train-006627.png train-008185.png train-008547.png train-009327.png (b) I trained the SVM with 20 values of C . The training error and test error are depicted below: 1 training error test error 0.1 error 0.01 0.001 1e-04 1e-05 1e-11 1e-10 1e-09 1e-08 C 1e-07 1e-06 1e-05 Cite as: Tommi Jaakkola, course materials for 6.867 Machine Learning, Fall 2006. MIT OpenCourseWare (http://ocw.mit.edu/), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY]. Notice that, except for exceptionally small values of C in the figure, the training error is 0 for this data set (it is easily separated by a hyper-plane). Observe also that the training error simply decreases as C increases; consequently, the best training error is achieved for very large C (in this case, it is zero training error). In contrast, if we could choose C with hindsight, i.e., measure the test error before having to select C , we would not pick a very large value. (c) We produce an analogous graph for SVMs trained on the partially mislabeled data. 1 training error test error error 0.1 0.01 0.001 1e-04 1e-11 1e-10 1e-09 1e-08 C 1e-07 1e-06 1e-05 Clearly now, having an intermediate value of C very slightly helps the training error and significantly reduces the test error. Again, though we could not legitimately select C based on the test error, we can expect similar results from cross-validation. Cite as: Tommi Jaakkola, course materials for 6.867 Machine Learning, Fall 2006. MIT OpenCourseWare (http://ocw.mit.edu/), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY]....
View Full Document

This document was uploaded on 03/20/2014 for the course EECS 6.867 at MIT.

Ask a homework question - tutors are online