View the step-by-step solution to:

Question

We consider the problem of learning a vector-valued function

data-value="R^dtoR^p">RdRp from input-output training data (xi,yi)i=1n where each xi is

a d-dimensional vector and each yi is a p-dimensional vector. We choose our hypothesis class to be the set of linear functions from Rd to Rp , that is function

satisfying f(x)=Wx for some d×p regression matrix W , and we want to minimize the squared error loss function

form.PNG

over the training data.

Let W be the minimizer of the empirical risk:

W=argminWRd×pJ(W)

-Derive a closed-form solution for W as a function of the data matrices XRn×d $ and YRn×p

-Show that solving the problem from the previous question is equivalent to independently solving p independent classical linear regression problems (one for each component of the output vector), and give an example of a multivariate regression task where performing independent regressions for each output variables is not the best thing to do.

-The low rank regression algorithm addresses the issue described in the previous question by imposing a low rank constraint on the regression matrix $W$. Intuitively, the low rank constraint encourages the model to capture linear dependencies in the components of the output vector.

Propose an algorithm to minimize the squared error loss over the training data subject to a low rank constraint on the regression matrix W :

minWRd×pJ(W) rank(W)  R

(hint: There are different ways to do that. Leverage the fact that rank(W) R if and only if there exists ARd×R $ and BRR×p $ such that W=AB )

form.PNG

Recently Asked Questions

Why Join Course Hero?

Course Hero has all the homework and study help you need to succeed! We’ve got course-specific notes, study guides, and practice tests along with expert tutors.

  • -

    Study Documents

    Find the best study resources around, tagged to your specific courses. Share your own to gain free Course Hero access.

    Browse Documents
  • -

    Question & Answers

    Get one-on-one homework help from our expert tutors—available online 24/7. Ask your own questions or browse existing Q&A threads. Satisfaction guaranteed!

    Ask a Question
Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes