lecture4 - CSE 6740 Lecture 4 How Do I Learn Any Density?...

Info iconThis preview shows pages 1–11. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: CSE 6740 Lecture 4 How Do I Learn Any Density? (Nonparametric Estimation) Alexander Gray agray@cc.gatech.edu Georgia Institute of Technology CSE 6740 Lecture 4 p. 1/3 6 Today 1. Nonparametric estimation (What if I dont want to specify a simple parametric form?) 2. Kernel density estimation (How can I estimate a density nonparametrically?) CSE 6740 Lecture 4 p. 2/3 6 Nonparametric Estimation What if I dont want to specify a simple parametric form? CSE 6740 Lecture 4 p. 3/3 6 Nonparametric Estimation What exactly do we mean by nonparametric? Example of a nonparametric model class, called a Sobolev space : F = braceleftbigg f : integraldisplay ( f ( x )) 2 dx < bracerightbigg (1) Nonparametric doesnt mean there are no parameters. There is typically a local model. It refers to model classes, like the one above, which arent parametric (having finite number of parameters). We sometimes say such a class is distribution-free . CSE 6740 Lecture 4 p. 4/3 6 Nonparametric Estimation A nonparametric method is one for which we can pretend the model class is actually such a class, as far as its asymptotic properties. In other words, it is a method for which one can show something like consistency with respect to a very general class of distributions (we want to say any distribution but this is of course never quite true). CSE 6740 Lecture 4 p. 5/3 6 Examples of Nonparametric Methods Some examples of popular nonparametric methods: Histogram, kernel density estimation (density estimation) Splines, wavelet regression (regression) Kernel discriminant analysis, nearest neighbor, support vector machines (classification) CSE 6740 Lecture 4 p. 6/3 6 Histogram Perhaps the simplest nonparametric density estimator is the histogram : hatwide f N ( x ) = m summationdisplay j =1 hatwide p j h I ( x B j ) (2) where h = 1 /m is the binwidth , Y j is the number of observations in bins B 1 = bracketleftbig , 1 m ) ,B 1 = bracketleftbig 1 m , 2 m ) ,... , hatwide p j = Y j /N , and p j = integraltext B j f ( u ) du . CSE 6740 Lecture 4 p. 7/3 6 Histogram CSE 6740 Lecture 4 p. 8/3 6 Histogram CSE 6740 Lecture 4 p. 9/3 6 Histogram Note a few things. First, the placement of the bins ( i.e. shifting a bit to the left or right) can make a significant qualitative difference. Second, the density estimate is not smooth....
View Full Document

Page1 / 31

lecture4 - CSE 6740 Lecture 4 How Do I Learn Any Density?...

This preview shows document pages 1 - 11. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online