assignment (2)

assignment (2) - Instructor Angela Yu Department of...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
Instructor Angela Yu Department of Cognitive Science UC San Diego Due: 01/29/09 COGS 118A: ASSIGNMENT 1 Problem 1. MLE for probabilistic linear regression Given a linear basis function model for regression, ¯ y ( x ) = w 0 + m j =1 w j φ j ( x ), and additive Gaussian noise, y ∼ N y, σ 2 ), show that the maximum likelihood estimate (MLE) for w 0 is: ˆ w * 0 = 1 n n X i =1 y i - m X j =1 w j ¯ φ j (1-1) where ¯ φ j = 1 n n i =1 φ j ( x ). That is, w 0 compensates for the difference between the sample mean of y and the weighted sample mean of the basis functions (each evaluated at all the data points). Problem 2. Linear basis function models of regression (a) Minimizing least squares error in regression leads to a nice analytical solution (Bishop Eq. 3.15). However, inverting a matrix can be computationally intense. Use tic and toc in Matlab to measure the amount of time it takes to invert a random matrix of size k x k generated with rand(k) , for different values of k . Start with some small value like 500 or 1000, and then increase it up to a point when Matlab hangs so long that you lose patience – note the value of k when this happens. Try at least 10 random matrices of each size, and average the computation time returned by tic and toc . Plot the mean duration against k . (b) If we use the regularized loss function: L λ ( w ) = ( Φw - y ) T ( Φw - y ) + λ w T w (2-1) show that the optimal w is: ˆ w * = ( Φ T Φ + λ I ) - 1 Φ T y (2-2) Hint: revisit lecture notes where I derived the optimal w for the un-regularized basic form of least squares error. Problem
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 3

assignment (2) - Instructor Angela Yu Department of...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online