18-classification2

18-classification2 - Classification II CS273 - Data and...

Info iconThis preview shows pages 1–10. Sign up to view the full content.

View Full Document Right Arrow Icon
Classification II S273 ata and Knowledge Bases CS273 - Data and Knowledge Bases Xifeng Yan Computer Science niversity of California at Santa Barbara University of California at Santa Barbara
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Department of Computer Science Announcements Readings: Christopher J. C. Burges, 1998. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery , 2 (2), 121–167. Data and Knowledge Bases | University of California at Santa Barbara 2
Background image of page 2
Department of Computer Science Classification: A Mathematical Mapping Classification: redicts categorical class labels predicts categorical class labels E.g., Personal homepage classification x = (x 1 , x 2 , x 3 , …), y i = +1 or 1 x 1 : # of a word “homepage” x 2 : # of a word “welcome” Mathematically = n y = {+1, } x X , y Y {1 , 1} We want to find a function f: X Y Data and Knowledge Bases | University of California at Santa Barbara 3
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Department of Computer Science Linear Classification Binary Classification problem Examples: SVM, Perceptron, Probabilistic Classifiers x x x x x • Vector: x, w • Scalar: y f(x) =0 x x x x o o Input: {( x 1 , y 1 ), …} Output: classification function f( x ) x o o o o o o o oo o f( x i ) > 0 for y i = +1 f( x i ) < 0 for y i = -1 o f( x ) =>w t x + b = 0 or w 1 x 1 +w 2 x 2 +b = 0 Data and Knowledge Bases | University of California at Santa Barbara 4
Background image of page 4
Department of Computer Science Dummy Attribute troduce a dummy attribute x x x x x t +b = 0 Introduce a dummy attribute X 0 =1 x x x x o o w x + b 0 ’) ’=0 x o o o o o o o oo o (w ) t x = 0, where, x’ = (x, 1), w’=(w, b) o f(x) => w t x Data and Knowledge Bases | University of California at Santa Barbara 5
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Department of Computer Science Perceptron Error For those misclassified x k f( x k ) < 0 while y k = +1 0 t y x w E x x x x x f( x k ) > 0 while y k = -1 k k k x x x x o o Gradient Decent E w w x o o o o o o o oo o o k k y x E Data and Knowledge Bases | University of California at Santa Barbara 6
Background image of page 6
Department of Computer Science Perceptron: Decision Plane Update E w w k k x y E k k x y w w k If we predict it incorrectly, Update one by one Update in a batch k k k x y w w Data and Knowledge Bases | University of California at Santa Barbara 7
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Department of Computer Science Perceptron Demo Step 1 Step 2 Data and Knowledge Bases | University of California at Santa Barbara 8 Step 3 Step 4
Background image of page 8
Department of Computer Science Perceptron Algorithm Initialize: w or each sample For each sample Predict the label of instance x to be y’ = sign{w t x} y’ , update the weight vector to If y y, update the weight vector to w = w + η yx lse w does not change Else w does not change Repeat until converge Data and Knowledge Bases | University of California at Santa Barbara 9
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 10
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/09/2012 for the course CS CS273 taught by Professor Xifengyan during the Spring '11 term at UCSB.

Page1 / 26

18-classification2 - Classification II CS273 - Data and...

This preview shows document pages 1 - 10. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online