Lecture24HO - CS440/ECE448: Intro to Articial Intelligence!...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon
Lecture 24: Perceptrons Prof. Julia Hockenmaier [email protected] http://cs.illinois.edu/fa11/cs440 CS440/ECE448: Intro to ArtiFcial Intelligence Regression Linear regression Given some data {(x,y)…}, with x, y ˥ R, Fnd a function f(x) = w 1 x + w 0 such that f(x) = y. o o o o o Squared Loss We want to Fnd a weight vector w which minimizes the loss (error) on the training data {(x 1 ,y 1 )…(x N , y N )} 4 CS440/ECE448: Intro AI L ( w ) = L 2 ( f w ( x i ), i = 1 N ! y i ) = ( y i " f w ( x i ) i = 1 N ! ) 2
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Linear regression We need to minimize the loss on the training data: w = argmin w Loss(f w ) We need to set partial derivatives of Loss(f w ) with respect to w1, w0 to zero. This has a closed-form solution for linear regression (see book). Gradient descent In general, we won ʼ t be able to Fnd a closed- form solution, so we need an iterative (local search) algorithm. We will start with an initial weight vector w, and update each element iteratively in the direction of its gradient: w i := w i ! d/dw i Loss( w ) 6 CS440/ECE448: Intro AI Binary classifcation with Naïve Bayes ±or each item x = (x 1…. x d ) , we compute f k ( x ) = P( x | C k )P(C k ) = P(C k ) " i P(x i |C k ) for both class C 1 and C 2 We assign class C 1 to x if f 1 ( x ) > f 2 ( x ) Equivalently, we can deFne a ʻ discriminant function ʼ f( x ) = f 1 ( x ) - f 2 ( x )
Background image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 10/13/2011 for the course CS 440 taught by Professor Levinson,s during the Spring '08 term at University of Illinois, Urbana Champaign.

Page1 / 5

Lecture24HO - CS440/ECE448: Intro to Articial Intelligence!...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online