lecture3-annotated - 1 Eric Xing @ CMU, 2006-2008 1 Machine...

Info iconThis preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 1 Eric Xing @ CMU, 2006-2008 1 Machine Learning Machine Learning 10 10-701/15 701/15-781, Fall 2008 781, Fall 2008 Na Na ve ve Bayes Bayes Classifier Classifier Eric Xing Eric Xing Lecture 3, September 15, 2008 Reading: Chap. 4 CB and handouts Eric Xing @ CMU, 2006-2008 2 z Mailing list z Homework 1 out today 2 Eric Xing @ CMU, 2006-2008 3 Nearest-Neighbor Learning Algorithm z Learning is just storing the representations of the training examples in D . z Testing instance x : z Compute similarity between x and all examples in D . z Assign x the category of the most similar example in D . z Does not explicitly compute a generalization or category prototypes. z Also called: z Case-based learning z Memory-based learning z Lazy learning Eric Xing @ CMU, 2006-2008 4 Asymptotic Analysis z Condition risk: r k ( X , X NN ) z Test sample X z NN sample X NN z Denote the event X is class I as X I z Assuming k =1 z When an infinite number of samples is available, X NN will be so close to X 3 Eric Xing @ CMU, 2006-2008 5 Asymptotic Analysis, cont. z Recall conditional Bayes risk: z Thus the asymptotic condition risk z It can be shown that z This is remarkable, considering that the procedure does not use any information about the underlying distributions and only the class of the single nearest neighbor determines the outcome of the decision. This is called the MacLaurin series expansion Eric Xing @ CMU, 2006-2008 6 kNN is an instance of Instance-Based Learning z What makes an Instance-Based Learner? z A distance metric z How many nearby neighbors to look at? z A weighting function (optional) z How to relate to the local points? 4 Eric Xing @ CMU, 2006-2008 7 Euclidean Distance Metric z Or equivalently, z Other metrics: z L 1 norm: |x-x'| z L norm: max |x-x'| (elementwise ) z Mahalanobis: where is full, and symmetric z Correlation z Angle z Hamming distance, Manhattan distance z = i i i i x x x x D 2 2 ) ' ( ) ' , ( ) ' ( ) ' ( ) ' , ( x x x x x x D T = Eric Xing @ CMU, 2006-2008 8 1-Nearest Neighbor (kNN) classifier Sports Science Arts 5 Eric Xing @ CMU, 2006-2008 9 2-Nearest Neighbor (kNN) classifier Sports Science Arts Eric Xing @ CMU, 2006-2008 10 3-Nearest Neighbor (kNN) classifier Sports Science Arts 6 Eric Xing @ CMU, 2006-2008 11 5-Nearest Neighbor (kNN) classifier Sports Science Arts Eric Xing @ CMU, 2006-2008 12 Case Study: kNN for Web Classification z Dataset z 20 News Groups (20 classes) z Download :(http://people.csail.mit.edu/jrennie/20Newsgroups/) z 61,118 words, 18,774 documents z Class labels descriptions Eric Xing @ CMU, 2006-2008 7 Eric Xing @ CMU, 2006-2008 13 Experimental Setup z Training/Test Sets: z 50%-50% randomly split....
View Full Document

Page1 / 29

lecture3-annotated - 1 Eric Xing @ CMU, 2006-2008 1 Machine...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online