lecture18

# lecture18 - 4/15/2009 Bad line representation? COMPUTER...

This preview shows pages 1–4. Sign up to view the full content.

4/15/2009 1 COMPUTER SCIENCE 51 Spring 2009 cs51.seas.harvard.edu Prof. Greg Morrisett Prof. Ramin Zabih Bad line representation? -10 -5 0 5 10 -10 -5 0 5 10 x 2*x 3*x 4*x 5*x 6*x 7*x 8*x LS fitting and residuals 0 1 2 3 4 5 6 0 2 4 6 8 10 12 Time Mileage ( ) 2 Err( , ) i i i m b y mx b = - + Nice example graph 0 100 200 300 400 500 600 14.6 14.8 15 15.2 15.4 15.6 15.8 16 Code for sample data ( define data (list '(15.85 235) '(15.69 280) '(15.38 360) '(15.3 442) '(14.84 528))) ( define (data-err m b data) ( let* ([predict ( lambda (x) (+ (* m x) b))] [residual ( lambda (pt) (- (predict (car pt)) (cadr pt)))] [sse (map square (map residual data))]) (foldr + 0 sse))) Greedy algorithm To find the LS squares line: Start with some initial ( m , b ) Make a small change to ( m , b ) See if Err( m , b ) goes down If so, make that your new ( m , b ) Done when small changes don’t reduce Err( m , b )

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4/15/2009 2 Hillclimbing Sometimes called hillclimbing In our case, we want to find the bottom of the hill not the top Can do a series of 1D optimizations Find best slope and fix it Find best intercept and fix it Repeat until done There are many alternatives Why least squares? There are lots of sensible ways to compute goodness of fit Why do we always use least squares? This is a deep topic But we will point out two things that are special about least squares Fix m=2, change b -1 -0.5 0 0.5 1 2 4 6 8 10 12 14 16 b sum of squared erors Fix b=-0.5, change m 1.8 1.9 2 2.1 2.2 2.5 3 3.5 4 4.5 5 5.5 m Some error functions are easy You will notice that the squared error has a single minimum value The plots I just showed you demonstrate this is true in 1D But it’s true in 2D also 0 50 100 0 20 40 60 80 10 0 50 100 150 200 250 300 350 400 450 500 Consequences For error functions like this, if we get an answer where a small change doesn’t improve it, we know we are right! Some issue with what “small” means Our LS-hillclimbingmethod will always converge to the right answer By slowly rolling downhill Hard to analyze how long it will take
4/15/2009 3 Easy and hard error measures What makes this error function easy? It has a single minimum, and we can get to it from anywhere by rolling downhill Not all error functions have this property But, least squares does Need to think before coding! Why is an error function hard? An error function where we can get stuck if we roll downhill is a hard one Where we get stuck depends on where we start (i.e., initial guess/conditions) An error function is easy if the area “above it” has a certain shape No nooks and crannies Easy: convex Hard: non-convex

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 07/26/2009 for the course COMPUTERSC CS51 taught by Professor Gregmorrisett during the Spring '09 term at Harvard.

### Page1 / 8

lecture18 - 4/15/2009 Bad line representation? COMPUTER...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online