This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: CSE 6740 Lecture 4 How Do I Learn Any Density? (Nonparametric Estimation) Alexander Gray [email protected] Georgia Institute of Technology CSE 6740 Lecture 4 – p. 1/3 6 Today 1. Nonparametric estimation (What if I don’t want to specify a simple parametric form?) 2. Kernel density estimation (How can I estimate a density nonparametrically?) CSE 6740 Lecture 4 – p. 2/3 6 Nonparametric Estimation What if I don’t want to specify a simple parametric form? CSE 6740 Lecture 4 – p. 3/3 6 Nonparametric Estimation What exactly do we mean by “nonparametric”? Example of a nonparametric model class, called a Sobolev space : F = braceleftbigg f : integraldisplay ( f ′′ ( x )) 2 dx < ∞ bracerightbigg (1) “Nonparametric” doesn’t mean there are no parameters. There is typically a local “model”. It refers to model classes, like the one above, which aren’t parametric (having finite number of parameters). We sometimes say such a class is distributionfree . CSE 6740 Lecture 4 – p. 4/3 6 Nonparametric Estimation A nonparametric method is one for which we can pretend the model class is actually such a class, as far as its asymptotic properties. In other words, it is a method for which one can show something like consistency with respect to a very general class of distributions (we want to say “any distribution” but this is of course never quite true). CSE 6740 Lecture 4 – p. 5/3 6 Examples of Nonparametric Methods Some examples of popular nonparametric methods: Histogram, kernel density estimation (density estimation) Splines, wavelet regression (regression) Kernel discriminant analysis, nearest neighbor, support vector machines (classification) CSE 6740 Lecture 4 – p. 6/3 6 Histogram Perhaps the simplest nonparametric density estimator is the histogram : hatwide f N ( x ) = m summationdisplay j =1 hatwide p j h I ( x ∈ B j ) (2) where h = 1 /m is the binwidth , Y j is the number of observations in bins B 1 = bracketleftbig , 1 m ) ,B 1 = bracketleftbig 1 m , 2 m ) ,... , hatwide p j = Y j /N , and p j = integraltext B j f ( u ) du . CSE 6740 Lecture 4 – p. 7/3 6 Histogram CSE 6740 Lecture 4 – p. 8/3 6 Histogram CSE 6740 Lecture 4 – p. 9/3 6 Histogram Note a few things. First, the placement of the bins ( i.e. shifting a bit to the left or right) can make a significant qualitative difference. Second, the density estimate is not smooth....
View
Full
Document
 Fall '08
 Staff
 Nonparametric statistics, Kernel density estimation, Kernel density estimator, nonparametric model class

Click to edit the document details