{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

problemset1 - CS229 Problem Set#1 1 CS 229 Public Course...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
CS229 Problem Set #1 1 CS 229, Public Course Problem Set #1: Supervised Learning 1. Newton’s method for computing least squares In this problem, we will prove that if we use Newton’s method solve the least squares optimization problem, then we only need one iteration to converge to θ * . (a) Find the Hessian of the cost function J ( θ ) = 1 2 m i =1 ( θ T x ( i ) y ( i ) ) 2 . (b) Show that the first iteration of Newton’s method gives us θ = ( X T X ) - 1 X T vector y , the solution to our least squares problem. 2. Locally-weighted logistic regression In this problem you will implement a locally-weighted version of logistic regression, where we weight different training examples differently according to the query point. The locally- weighted logistic regression problem is to maximize ( θ ) = λ 2 θ T θ + m summationdisplay i =1 w ( i ) bracketleftBig y ( i ) log h θ ( x ( i ) ) + (1 y ( i ) ) log(1 h θ ( x ( i ) )) bracketrightBig . The λ 2 θ T θ here is what is known as a regularization parameter, which will be discussed in a future lecture, but which we include here because it is needed for Newton’s method to perform well on this task. For the entirety of this problem you can use the value λ = 0 . 0001. Using this definition, the gradient of ( θ ) is given by θ ( θ ) = X T z λθ where z R m is defined by z i = w ( i ) ( y ( i ) h θ ( x ( i ) )) and the Hessian is given by H = X T DX λI where D R m × m is a diagonal matrix with D ii = w ( i ) h θ ( x ( i ) )(1 h θ ( x ( i ) )) For the sake of this problem you can just use the above formulas, but you should try to derive these results for yourself as well. Given a query point x , we choose compute the weights w ( i ) = exp parenleftbigg || x x ( i ) || 2 2 τ 2 parenrightbigg .
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}