SVM-PR2010_387005365 - Support Vector Machine 1 Outline...

Info iconThis preview shows pages 1–20. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Support Vector Machine Support Vector Machine 张长水 清华大学自动化系
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Outline z Linearly separable patterns z Linearly non-separable patterns z Nonlinear case z Some examples
Background image of page 2
3 Linearly separable case Optimal Separating hyperplane
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 Optimal Hyperplane
Background image of page 4
5 Linear classfication N i i i d 1 )} , {( set sample Training = = x T 1, positive patterns 1,negative patterns i i d d =+ =− 0 T wx b += Decision surface: 0 for 1 1 T ii T d d + ≥= + + <=
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6 Decision surface (line) figure copied from reference [4]
Background image of page 6
7 Measure of distance o p o w xx r w =+ () T oo o g xw x br w = Decomposition of x: o gx r w =
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
8 Linear classfication N i i i d 1 )} , {( set sample Training = = x T 1, positive patterns 1,negative patterns i i d d =+ =− 0 T oo wx b + = 0 for 1 1 T oi o i T o i d d +≥ = + +< = Decision surface: 1 for 1 for 1 T o i T o i d d + ≥+ + ≤−
Background image of page 8
9 Margin of separation () s x ( ) 1 for 1 sT s s oo gx wx b d = += ± = ± s o g x r w = Consider a support vector then 1 if 1 w 1 if 1 w s o s o d d = + = = − Margin of separation 2 2 o r w ρ ==
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
10 Optimization problem ( ) 1 1,2,. .., T ii db f o r i N +≥ = wx 1 min ( ) 2 T f = ww w 1 1 T bf o r d T o r d + ≥+ =+ + ≤− =− N i i i d 1 )} , {( set sample Training = = x T subject to (P) (w)= ( ) 1 0 .., T i g for i N +− =
Background image of page 10
11 Lagrange function (, , ) Jwb α () f w i gw 1 1 (,, ) [ ( ) 1 ] 2 N TT ii i i Jb d b αα = = −+ ww w w x Lagrange multipliers find saddle point of what? why? exist? ) w Φ
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
12 A saddle point saddle point
Background image of page 12
13 What is a saddle point nm wD R E R α ∈⊂ : DE R Φ ×→ Definition: saddle point '' (, ) ) ( , ) , ww w w D E ααα Φ ≤Φ ∀ ∈ variables function ) w
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
14 why we find a saddle point '' (, ) w α 1 (,) () N ii i wf w g w αα = Φ= if is a saddle point of then ' w is a solution of (P) Theorem: Proof: ' ' ' ' 11 1 ( ) ( ) NN N i f w g w f w g w f w g w == = −≤ ∑∑ ' ' ' ' ' 1 ()0 ( )()0 N i i i i fw gw = −+ ' 2 2 1, , ... let =+ = = ' i ' w [3] is a feasible solution of (P)
Background image of page 14
15 why we find a saddle point (ctd.) '' ' ' ' ' 11 1 () ( ) ( ) NN N ii i fw gw αα α == = −≤ ∑∑ let 0 α= 1 ()0 N i = 1 0 N i = = consider the second inequality 1 ( ) ( ) N i f w f w g w = ≤− ( ) 0 if is feasible i w ' ( ) f wf w ' w is optimal solution of (P)
Background image of page 15

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
16 Strong Duality max min ( , ) min max ( , ) ww αα α Φ Strong Duality: the condition holds if and only if there exists a pair '' (, ) w satisfies the saddle-point condition for Φ Proof: (omitted) “Stephen G.Nash & Ariela Sofer Linear and Nonlinear Programming” pp468 maxmin ( , ) ( , ) minmax ( , ) w Φ= Φ = Φ
Background image of page 16
17 Dual Problem 00 1 () m a x (, ) m a x [() ] N ii i Lw w f w g w αα ≥≥ = = min ( ) w m i n ) w Qw α = Φ primal function primal problem dual function dual problem 0 max ( ) Q 0 min max ( , ) w w Φ 0 max min ( , ) w w Φ we prefer to solve the dual problem!
Background image of page 17

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
18 Solve the dual problem 1 1 (,, ) [ ( ) 1 ] 2 N TT ii i i Jb d b αα = = −+ ww w w x , () m i n (, , ) wb QJ w b α = ) w Φ= dual function: , ) 0 Jwb w = 1 N iii i wd x = = , ) 0 b = 1 0 N i d = =
Background image of page 18
19 1 1 (,, ) [ ( ) 1 ] 2 N TT ii i i Jb d b αα = =− +
Background image of page 19

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 20
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 06/02/2010 for the course ELECTRONIC PC2010S taught by Professor Zhangchangshui during the Spring '10 term at Tsinghua University.

Page1 / 66

SVM-PR2010_387005365 - Support Vector Machine 1 Outline...

This preview shows document pages 1 - 20. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online