class_10_26

class_10_26 - Statistical Data Mining ORIE 474 Fall 2007...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Statistical Data Mining ORIE 474 Fall 2007 Tatiyana Apanasovich 10/26/07 Nearest Neighbor Models
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
10.6 Nearest Neighbor (NN) Models Data: (x(i),c(i)), where i=1,…,n and c(i) in {c 1 ,…, c m } Distance function: d(x(i),x(j)) Model Structure: To classify a new object x 0 : 1. we examine the k closest points (nearest neighbors) to x 0 in the training data set. Denote them by x(i 1 ),…,x(i k ). 2. Assign the object to the class that has the majority of the points among these k
Background image of page 2
NN (cont’d) What means “k closest points to x 0 “? Think of a small volume of the space of input variables X, centered at x 0 , with the radius the distance to the k th nearest neighbor Subspace of variables in training data, centered at x 0 , with radius r: Distance to k th nearest neighbor r} ) d(x(i),x {x(i) x D r = 0 0 : data training ) ( } | ) ( :| min{ ) , ( 0 0 k x D r k x r r =
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
NN Model Structure ML estimator of the probability that a point in this small volume belongs to class c j : where (if there are more than one data points in the training set with equal distance to x 0 as a k th NN, choose at random which to include in D k (x 0 )) Hence, we have = = ) ( ) ( 0 , 0 ) ) ( ( 1 1 ) ( ˆ x D i x j k j k c i c k x p ) ( ˆ max arg where , ˆ 0 k j, ,..., 1 * 0 x p j* c c m j j = = = ) ( ) ( 0 ) , ( 0 0 x D x D k x r k =
Background image of page 4
NN (cont’d) Note: The MLE of the probability that a point in this small volume belongs to a certain class is
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 15

class_10_26 - Statistical Data Mining ORIE 474 Fall 2007...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online