class08

class08 - Chapter6.ClassificationandPrediction...

Info iconThis preview shows pages 1–10. Sign up to view the full content.

View Full Document Right Arrow Icon
August 29, 2011 Data Mining: Concepts and Techniques 1 Chapter 6. Classification and Prediction What is classification? What is  prediction? Issues regarding classification  and prediction Classification by decision tree  induction Bayesian classification Rule-based classification Classification by back  propagation Support Vector Machines  (SVM)  Associative classification  Lazy learners (or learning from  your neighbors) Other classification methods Prediction Accuracy and error measures Ensemble methods Model selection
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
August 29, 2011 Data Mining: Concepts and Techniques 2 Nearest Neighbor Classifiers Basic idea: If it walks like a duck, quacks like a duck, then  it s probably a duck Training Records Test Record Compute Distance Choose k of the “nearest” records
Background image of page 2
August 29, 2011 Data Mining: Concepts and Techniques 3 Nearest neighbor method Majority vote within the k nearest neighbors     K= 1: brown K= 3: green new
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
August 29, 2011 Data Mining: Concepts and Techniques 4 Nearest-Neighbor Classifiers Requires three things The set of stored records Distance Metric to compute distance between records The value of k , the number of nearest neighbors to retrieve To classify an unknown record: Compute distance to other training records Identify k nearest neighbors Use class labels of nearest neighbors to determine the class label of unknown record (e.g., by taking majority vote) Unknown record
Background image of page 4
August 29, 2011 Data Mining: Concepts and Techniques 5 Definition of Nearest Neighbor X X X (a) 1-nearest neighbor (b) 2-nearest neighbor (c) 3-nearest neighbor K-nearest neighbors of a record x are data points that have the k smallest distance to x
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
August 29, 2011 Data Mining: Concepts and Techniques 6 1 nearest-neighbor Voronoi Diagram
Background image of page 6
August 29, 2011 Data Mining: Concepts and Techniques 7 Nearest Neighbor Classification Compute distance between two points: Euclidean distance  Determine the class from nearest neighbor list take the majority vote of class labels among the  k-nearest neighbors Weigh the vote according to distance  weight factor, w = 1/d 2 - = i i i q p q p d 2 ) ( ) , (
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
August 29, 2011 Data Mining: Concepts and Techniques 8 Nearest Neighbor Classification… Choosing the value of k: If k is too small, sensitive to noise points If k is too large, neighborhood may include points from  other classes X
Background image of page 8
August 29, 2011 Data Mining: Concepts and Techniques 9 Nearest Neighbor Classification… Scaling issues Attributes may have to be scaled to prevent  distance measures from being dominated by  one of the attributes Example:  height of a person may vary from 1.5m to 1.8m  weight of a person may vary from 90lb to 300lb  income of a person may vary from $10K to $1M
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 10
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 53

class08 - Chapter6.ClassificationandPrediction...

This preview shows document pages 1 - 10. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online