{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

SVM-PR2010_387005365

# SVM-PR2010_387005365 - Support Vector Machine 1 Outline...

This preview shows pages 1–20. Sign up to view the full content.

1 Support Vector Machine Support Vector Machine 张长水 清华大学自动化系

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 Outline z Linearly separable patterns z Linearly non-separable patterns z Nonlinear case z Some examples
3 Linearly separable case Optimal Separating hyperplane

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4 Optimal Hyperplane
5 Linear classfication N i i i d 1 )} , {( set sample Training = = x T 1, positive patterns 1,negative patterns i i d d =+ =− 0 T wx b += Decision surface: 0 for 1 1 T ii T d d + ≥= + + <=

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
6 Decision surface (line) figure copied from reference [4]
7 Measure of distance o p o w xx r w =+ () T oo o g xw x br w = Decomposition of x: o gx r w =

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
8 Linear classfication N i i i d 1 )} , {( set sample Training = = x T 1, positive patterns 1,negative patterns i i d d =+ =− 0 T oo wx b + = 0 for 1 1 T oi o i T o i d d +≥ = + +< = Decision surface: 1 for 1 for 1 T o i T o i d d + ≥+ + ≤−
9 Margin of separation () s x ( ) 1 for 1 sT s s oo gx wx b d = += ± = ± s o g x r w = Consider a support vector then 1 if 1 w 1 if 1 w s o s o d d = + = = − Margin of separation 2 2 o r w ρ ==

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
10 Optimization problem ( ) 1 1,2,. .., T ii db f o r i N +≥ = wx 1 min ( ) 2 T f = ww w 1 1 T bf o r d T o r d + ≥+ =+ + ≤− =− N i i i d 1 )} , {( set sample Training = = x T subject to (P) (w)= ( ) 1 0 .., T i g for i N +− =
11 Lagrange function (, , ) Jwb α () f w i gw 1 1 (,, ) [ ( ) 1 ] 2 N TT ii i i Jb d b αα = = −+ ww w w x Lagrange multipliers find saddle point of what? why? exist? ) w Φ

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
13 What is a saddle point nm wD R E R α ∈⊂ : DE R Φ ×→ Definition: saddle point '' (, ) ) ( , ) , ww w w D E ααα Φ ≤Φ ∀ ∈ variables function ) w

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
14 why we find a saddle point '' (, ) w α 1 (,) () N ii i wf w g w αα = Φ= if is a saddle point of then ' w is a solution of (P) Theorem: Proof: ' ' ' ' 11 1 ( ) ( ) NN N i f w g w f w g w f w g w == = −≤ ∑∑ ' ' ' ' ' 1 ()0 ( )()0 N i i i i fw gw = −+ ' 2 2 1, , ... let =+ = = ' i ' w [3] is a feasible solution of (P)
15 why we find a saddle point (ctd.) '' ' ' ' ' 11 1 () ( ) ( ) NN N ii i fw gw αα α == = −≤ ∑∑ let 0 α= 1 ()0 N i = 1 0 N i = = consider the second inequality 1 ( ) ( ) N i f w f w g w = ≤− ( ) 0 if is feasible i w ' ( ) f wf w ' w is optimal solution of (P)

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
16 Strong Duality max min ( , ) min max ( , ) ww αα α Φ Strong Duality: the condition holds if and only if there exists a pair '' (, ) w satisfies the saddle-point condition for Φ Proof: (omitted) “Stephen G.Nash & Ariela Sofer Linear and Nonlinear Programming” pp468 maxmin ( , ) ( , ) minmax ( , ) w Φ= Φ = Φ
17 Dual Problem 00 1 () m a x (, ) m a x [() ] N ii i Lw w f w g w αα ≥≥ = = min ( ) w m i n ) w Qw α = Φ primal function primal problem dual function dual problem 0 max ( ) Q 0 min max ( , ) w w Φ 0 max min ( , ) w w Φ we prefer to solve the dual problem!

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
18 Solve the dual problem 1 1 (,, ) [ ( ) 1 ] 2 N TT ii i i Jb d b αα = = −+ ww w w x , () m i n (, , ) wb QJ w b α = ) w Φ= dual function: , ) 0 Jwb w = 1 N iii i wd x = = , ) 0 b = 1 0 N i d = =
19 1 1 (,, ) [ ( ) 1 ] 2 N TT ii i i Jb d b αα = =− +

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 66

SVM-PR2010_387005365 - Support Vector Machine 1 Outline...

This preview shows document pages 1 - 20. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online