Stat841f09 - Wiki Course Notes

# We have to control its complexity so that it does not

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: nits than input units, we can essentially project to a higher- dimensional space, as we did in our earlier trick. However, this does not mean that an RBF network will actually do this, it is merely a way to convince yourself that RBF networks (and neural networks) can fit arbitrary models. Nevertheless, it is also noticed that just wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 49/74 10/09/2013 Stat841 - Wiki Cour se Notes because of such great power, the problem of overfitting appears more important. We have to control its complexity so that it does not fit to an arbitrary training model but to a general one. RBF networks for clas s ification -- a probabilis tic paradigm An RBF network is akin to fitting a Gaussian mixture model to data. We assume that each class can be modelled by a single function and data is generated by a mixture model. According to Bayes Rule, While all classifiers that we have seen thus far in the course have been in discriminative form, the RBF network is a generative model that can be represented using a directed graph. We can replace the class conditional density in the above conditional probability expression by marginalizing Figure 1: RBF graphical model over : Note We made the assumption that each class can be modelled by a single function model has the form: where are mixing proportions, and that the data was generated by a mixture model. The Gaussian mixture , and and are the mean and covariance of each Gaussian density respectively. [14] The generative model in Figure 1 shows graphically how each Gaussian in the mixture model is chosen to sample from. Radial Basis Function (RBF) Networks - November 9th, 2009 RBF Network for clas s ification (A probabilis tic point of view) Using RBF Network[25] (http://en.wikipedia.org/wiki/Radial_basis_function_network) to do classification, we usually treat it as regression problem. We want to set a threshold to decide what the data’s class membership is. However, to find some insight to the classification problem of what we are doing in terms of RBF Network, we often think of mixture models and make certain assumptions. Before...
View Full Document

## This document was uploaded on 03/07/2014.

Ask a homework question - tutors are online