{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

lecture4

# lecture4 - CSE 6740 Lecture 4 How Do I Learn Any...

This preview shows pages 1–11. Sign up to view the full content.

CSE 6740 Lecture 4 How Do I Learn Any Density? (Nonparametric Estimation) Alexander Gray [email protected] Georgia Institute of Technology CSE 6740 Lecture 4 – p. 1/3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Today 1. Nonparametric estimation (What if I don’t want to specify a simple parametric form?) 2. Kernel density estimation (How can I estimate a density nonparametrically?) CSE 6740 Lecture 4 – p. 2/3
Nonparametric Estimation What if I don’t want to specify a simple parametric form? CSE 6740 Lecture 4 – p. 3/3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Nonparametric Estimation What exactly do we mean by “nonparametric”? Example of a nonparametric model class, called a Sobolev space : F = braceleftbigg f : integraldisplay ( f ′′ ( x )) 2 dx < bracerightbigg (1) “Nonparametric” doesn’t mean there are no parameters. There is typically a local “model”. It refers to model classes, like the one above, which aren’t parametric (having finite number of parameters). We sometimes say such a class is distribution-free . CSE 6740 Lecture 4 – p. 4/3
Nonparametric Estimation A nonparametric method is one for which we can pretend the model class is actually such a class, as far as its asymptotic properties. In other words, it is a method for which one can show something like consistency with respect to a very general class of distributions (we want to say “any distribution” but this is of course never quite true). CSE 6740 Lecture 4 – p. 5/3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Examples of Nonparametric Methods Some examples of popular nonparametric methods: Histogram, kernel density estimation (density estimation) Splines, wavelet regression (regression) Kernel discriminant analysis, nearest neighbor, support vector machines (classification) CSE 6740 Lecture 4 – p. 6/3
Histogram Perhaps the simplest nonparametric density estimator is the histogram : hatwide f N ( x ) = m summationdisplay j =1 hatwide p j h I ( x B j ) (2) where h = 1 /m is the binwidth , Y j is the number of observations in bins B 1 = bracketleftbig 0 , 1 m ) , B 1 = bracketleftbig 1 m , 2 m ) , . . . , hatwide p j = Y j /N , and p j = integraltext B j f ( u ) du . CSE 6740 Lecture 4 – p. 7/3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Histogram CSE 6740 Lecture 4 – p. 8/3
Histogram CSE 6740 Lecture 4 – p. 9/3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Histogram Note a few things. First, the placement of the bins ( i.e. shifting a bit to the left or right) can make a significant qualitative difference. Second, the density estimate is not smooth.
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}