problemset1

problemset1 - CS229 Problem Set #1 1 CS 229, Public Course...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
CS229 Problem Set #1 1 CS 229, Public Course Problem Set #1: Supervised Learning 1. Newton’s method for computing least squares In this problem, we will prove that if we use Newton’s method solve the least squares optimization problem, then we only need one iteration to converge to θ * . (a) Find the Hessian of the cost function J ( θ ) = 1 2 m i =1 ( θ T x ( i ) y ( i ) ) 2 . (b) Show that the ±rst iteration of Newton’s method gives us θ = ( X T X ) - 1 X T v y , the solution to our least squares problem. 2. Locally-weighted logistic regression In this problem you will implement a locally-weighted version of logistic regression, where we weight di²erent training examples di²erently according to the query point. The locally- weighted logistic regression problem is to maximize ( θ ) = λ 2 θ T θ + m s i =1 w ( i ) b y ( i ) log h θ ( x ( i ) ) + (1 y ( i ) ) log(1 h θ ( x ( i ) )) B . The λ 2 θ T θ here is what is known as a regularization parameter, which will be discussed in a future lecture, but which we include here because it is needed for Newton’s method to perform well on this task. For the entirety of this problem you can use the value λ = 0 . 0001. Using this de±nition, the gradient of ( θ ) is given by θ ( θ ) = X T z λθ where z R m is de±ned by z i = w ( i ) ( y ( i ) h θ ( x ( i ) )) and the Hessian is given by H = X T DX λI where D R m × m is a diagonal matrix with D ii = w ( i ) h θ ( x ( i ) )(1 h θ ( x ( i ) )) For the sake of this problem you can just use the above formulas, but you should try to derive these results for yourself as well. Given a query point x , we choose compute the weights w ( i ) = exp p || x x ( i ) || 2 2 τ 2 P . Much like the locally weighted linear regression that was discussed in class, this weighting
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/24/2010 for the course CS 229 at Stanford.

Page1 / 4

problemset1 - CS229 Problem Set #1 1 CS 229, Public Course...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online