lecture14-SVMs-handout-6-per

Eachnonzeroiindicatesthatcorrespondingxiisasupportvect

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: distance
from
the
hyperplane
is
   The
margin
is:
 7
 Sec. 15.1 Introduc)on to Informa)on Retrieval   Hyperplane

 







wT
x
+
b
=
0
 wTxa + b = 1 wTxb + b = -1   Then
we
can
formulate
the
quadra)c op)miza)on problem: Find w and b such that is maximized; and for all {(xi , yi)}   Extra
scale
constraint:
 







mini=1,…,n
|wTxi
+
b|
=
1
   This
implies:
 







wT(xa–xb)
=
2
 
 
ρ
=
2/||w||2
 Sec. 15.1 Introduc)on to Informa)on Retrieval Linear
SVMs
Mathema)cally
(cont.)
 Linear
Support
Vector
Machine
(SVM)
 ρ 8
 wTx i + b ≥ 1 if yi=1; wTxi + b ≤ -1 if yi = -1   A
berer
formula)on
(min
||w||
=
max
1/
||w||
):

 wT x + b = 0 Find w and b such that Φ(w) =½ wTw is minimized; 9
 Introduc)on to Informa)on Retrieval Sec. 15.1 Solving
the
Op)miza)on
Problem
 yi (wTxi + b) ≥ 1 10
 Sec. 15.1 Introduc)on to Informa)on Retrieval The
Op)miza)on
Problem
Solu)on
   The
solu)on
has
the
form:

 Find w and b such that Φ(w) =½ wTw is minimized; and for all {(xi ,yi)}: yi (wTxi + b) ≥ 1 w =Σαiyixi   This
is
now
op)mizing
a
quadra)c func)on
subject
to
linear constraints
   Quadra)c
op)miza)on
problems
are
a
well‐known
class
of
mathema)cal
 programming
problem,
and
many
(intricate)
algorithms
exist
for
solving
them
 (with
many
special
ones
built
for
SVMs)
   The
solu)on
involves
construc)ng
a
dual problem where
a
Lagrange mul)plier
αi is
associated
with
every
constraint
in
the
primary
problem:
 Find α1…αN such that Q(α) =Σαi - ½ΣΣαiαjyiyjxiTxj is maximized and (1) Σαiyi = 0 (2) αi ≥ 0 for all αi and for all {(xi ,yi)}: 11
 b= yk- wTxk for any xk such that αk≠ 0   Each
non‐zero
αi
indicates
that
corresp...
View Full Document

This document was uploaded on 02/26/2014.

Ask a homework question - tutors are online