This preview shows page 1. Sign up to view the full content.
Unformatted text preview: le= Stat841&pr intable= yes 45/74 10/09/2013 Stat841 - Wiki Cour se Notes For example, suppose we want to fit a polynomial model to the data set and split the set into four equal subsets as shown in Figure 2. First we choose the degree to be 1,
i.e. a linear model. Next we use the first three sets as training sets and the last as validation set, then the 1st, 2nd, 4th subsets as training set and the 3rd as validation set, so
on and so forth until all the subsets have been the validation set once (all observations are used for both training and validation). After we get
, we can
calculate the average
for degree 1 model. Similarly, we can estimate the error for n degree model and generate a simulating curve. Now we are able to choose the right
degree which corresponds to the minimum error. Also, we can use this method to find the optimal unit number of hidden layers of neural networks. We can begin with 1 unit
number, then 2, 3 and so on and so forth. Then find the unit number of hidden layers with lowest average error. Generalized Cros s -validation
If the vector of observed values is denoted by , and the vector of fitted values by . ,
where the hat matrix is given by
, , Then the GCV approximation is given by
, Thus, one of the biggest advantages of the GCV is that the trace is more easily computed. Leave-one-out Cros s -validation
Leave- one- out cross- validation involves using all but one data point in the original training data set to train our model, then using the data point that we initially left out as a
means to estimate true error. But repeating this process for every data point in our original data set, we can obtain a good estimation of true error.
In other words, leave- one- out cross- validation is k- fold cross- validation in which we set the subset number to be the cardinality of the whole data set. In the above example, we can see that k- fold cross- validation can be computationally expensive: for every possible value of the parameter, we must...
View Full Document
- Winter '13