midterm2011_SOLUTION

midterm2011_SOLUTION - STANFORD UNIVERSITY CS 229, Autumn...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: STANFORD UNIVERSITY CS 229, Autumn 2011 Midterm Examination Wednesday, November 9, 6:00pm-9:00pm Question Points 1 Generalized Linear Models /15 2 Gaussian Naive Bayes /15 3 Linear Invariance of Logistic Regression /12 4 2-Regularized SVM /18 5 Uniform Convergence /16 6 Short Answers /38 Total /114 Name of Student: SUID: The Stanford University Honor Code: I attest that I have not given or received aid in this examination, and that I have done my share and taken an active part in seeing to it that others as well as myself uphold the spirit and letter of the Honor Code. Signed: CS229 Midterm 2 1. [15 points] Generalized Linear Models In class, we showed that the Bernoulli and Gaussian distributions are exponential family distributions, which are of the form p ( y ; ) = b ( y ) exp( T T ( y )- a ( )) In this problem, we will consider a different exponential family distribution, specifi- cally the Exponential distribution, which has a density given by p ( y ; ) = exp(- y ) Here, y 0 is a non-negative real number, and the distribution is parameterized by R . (a) [5 points] Write the Exponential distribution in the exponential family form given above. You will need to come up with expressions for , b ( y ), T ( y ), and a ( ). Answer: p ( y ; ) = exp(log - y ) b ( y ) = 1 =- T ( y ) = y a ( ) =- log (- ) Note: an equally valid solution has T ( y ) =- y, = , and a ( ) =- log( ) . This will give sign flips in parts b and c, but results in an identical Hessian in part d. CS229 Midterm 3 (b) [2 points] Derive the canonical response function g ( ), which gives the Expo- nential distributions mean as a function of the natural parameter . You may use the fact that an Exponential distribution (with parameter ) has mean 1 . Answer: g ( ) = 1 =- 1 (c) [2 points] Assuming that we have a training set { ( x (1) ,y (1) ) ,..., ( x ( m ) ,y ( m ) ) } of m independently and identically distributed (IID) examples, write down the log-likelihood ( ) of the parameters. Answer: L ( ) = m Y i =1 p ( y ( i ) | x ( i ) ; ) = m Y i =1 exp ( T T ( y ( i ) )- a ( ) ) = m Y i =1 exp ( T x ( i ) ) T y ( i ) + log (- T x ( i ) ) ( ) = m X i =1 T x ( i ) y ( i ) + log (- T x ( i ) ) (d) [6 points] Find the hessian H of the log-likelihood ( ), and show that it is negative semi-definite. CS229 Midterm 4 Answer: First, the gradient: ( ) j = m X i =1 j ( T x ( i ) y ( i ) + log (- T x ( i ) )) = m X i =1 x ( i ) j y ( i ) + j log (- T x ( i ) ) = m X i =1 x ( i ) j y ( i ) + 1- T x ( i ) j (- T x ( i ) ) = m X i =1 x ( i ) j y ( i ) + x ( i ) j T x ( i ) = m X i =1 y ( i ) + 1 T x ( i ) x ( i ) j Now, the Hessian: ( ) j = m X i =1 y ( i ) + 1 T x ( i ) x ( i ) j 2 j k = m X i =1 x ( i ) j k 1 T x ( i ) = m X i =1 x ( i ) j- 1 ( T x ( i...
View Full Document

Page1 / 24

midterm2011_SOLUTION - STANFORD UNIVERSITY CS 229, Autumn...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online