This preview shows page 1. Sign up to view the full content.
Unformatted text preview: s functions. Such a model follows the training data perfectly. However, the model is not representative features of the true underlying data source, and this is why it
fails to correctly model new data points.
In general, the training error rate will be less than the testing error on the new data. A model typically adapts to the training data, and hence the training error will be an overly
optimistic estimate of the testing error. An obvious way to well estimate testing err is to add a penalty term to the training error to compensate. Actually, SURE is developed
based on this. Stein's unbias ed ris k es timate (SURE)
Important Notation (http://e n.wikipe dia.org/wiki/Ste in's _unbias e d_ris k_e s timate )
denote the predict ion model, which is estimated from a training sample by the RBF neural network model.
denote the t rue model.
denote the t raining error,which is the average loss over the training sample.
wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 53/74 10/09/2013 Stat841 - Wiki Cour se Notes denote the t est error, which is the expected prediction error on an independent test sample.
denote the mean squared error, where is the estimated model and is the true model. The Bias- Variance Decomposition: Since, , which is equal to zero. Suppose the observations , where and is additive Gaussian noise .We need to estimate from training data set . Let ， then The last term can be written as:
, where and both have same mean . Ste in's Le mma (http://e n.wikipe dia.org/wiki/Ste in%27s _le mma)
If is and if is weakly differentiable,such that According to Stein's Lemma, the last cross term of
: Let Since . Then , then , , since is the true model, not the function of the observations can be written as
, and is a constant. So , then . . The derivation is as follows.
and is the variance in . . So, Two Diffe re nt Cas e s
SURE in RBF, Automatic basis selection for RBF networks using Stein’s unbiased risk estimator,Ali Ghodsi Dale Schuurmans
View Full Document
- Winter '13