Case Study 4: Collaborative Filtering
Collaborative Filtering
Matrix Completion
Alternating Least Squares
Machine Learning/Statistics for Big Data
CSE599C1/STAT592, University of Washington
Carlos Guestrin
February 28th, 2013
Carlos Guestrin 2013
1
Collab
Low-Rank Matrix Optimization Problems
Junxiao Song and Daniel P. Palomar
The Hong Kong University of Science and Technology (HKUST)
ELEC 5470 - Convex Optimization
Fall 2014-15, HKUST, Hong Kong
J. Song and D. Palomar (HKUST)
Low-Rank Optimization
ELEC 54
Low-rank Matrix Completion via
Convex Optimization
Ben Recht
Center for the Mathematics of Information
Caltech
Recommender Systems
Netflix Prize
One million big ones!
Given 100 million ratings on a scale of 1 to 5, predict 3
million ratings to highest a
EE/AA/ME 578
Univ. of Washington, Winter 2015
Homework 7
1. Matrix completion. In a movie recommendation system (e.g., the Netix challenge), users
watch a subset of movies and submit ratings. The recommender system stores a huge rating
matrix, whose rows
EE/AA/ME 578
Univ. of Washington, Winter 2015
Homework 7
1. Matrix completion. In a movie recommendation system (e.g., the Netix challenge), users
watch a subset of movies and submit ratings. The recommender system stores a huge rating
matrix, whose rows
Convex Analysis and
Optimization
Chapter 5 Solutions
Dimitri P. Bertsekas
with
Angelia Nedi and Asuman E. Ozdaglar
c
Massachusetts Institute of Technology
Athena Scientic, Belmont, Massachusetts
http:/www.athenasc.com
LAST UPDATE
April 15, 2003
CHAPTER 5:
ECE 490: Introduction to Optimization
Spring 2014
Problem Set I
R. Srikant
Quiz: Jan. 30
1.
(a)
Find the inf and sup of the function
f (x1 , x2 ) = x1 + 2x2
over the set
D = x : x2 + x2 1, x1 > 0, x2 > 0 .
1
2
(b)
Does there exist (i) a minimum and (ii) a
ECE 490: Introduction to Optimization
Spring 2014
Problem Set V
R. Srikant
Quiz: Feb. 27
1.
Consider the unconstrained optimization problem minxRn f (x). Assume that f is twice continuously dierentiable, and show that if x is a local min, then f (x ) = 0
ECE 490: Introduction to Optimization
Spring 2014
Problem Set X
R. Srikant
Quiz: May 1
1. Prove the following properties of the subgradient, where f and g are convex functions:
(a) Scaling: for a > 0, (a f ) = a f.
(b) Addition: (f + g) = f + g.
(c) Ane c
ECE 490: Introduction to Optimization
Spring 2014
Problem Set IX
R. Srikant
Quiz: Apr. 17
1. Let f and g be convex functions and S a convex set. Show that
D = cfw_(y, z, w) : Ax b = y, g (x) z, f (x) w, for some x S
is a convex set. Here A is a matrix and
ECE 490: Introduction to Optimization
Spring 2014
Problem Set XI
R. Srikant
1. Recall that the method of multipliers (for our example with quadratic f (x) =
and linear constraints, h (x) = Ax b), consists of iterating the two update steps:
ck
h (x)
x
2
=
ECE 490: Introduction to Optimization
Spring 2014
Problem Set VI
R. Srikant
Quiz: Mar. 13
1. (DB) Use the Lagrange multiplier theorem to solve the following problems:
2
n
i=1
a) f (x) = x , h (x) =
b) f (x) =
n
i=1
xi 1
xi , h (x) = x
2
2
1
T
c) f (x) = x
ECE 490: Introduction to Optimization
Spring 2014
Problem Set II
R. Srikant
Quiz: Feb. 6
1.
Determine if the following function is convex, concave, or neither.
1
f (x1 , x2 , x3 ) = 2x2 + x1 x3 + x2 + 2x2 x3 + x2
1
2
2 3
Solution:
The Hessian of f is
4
2
ECE 490: Introduction to Optimization
Spring 2014
Problem Set IV
R. Srikant
Quiz: Feb. 20
1.
2 1 0
Consider a symmetric matrix, A = 1 2 0 . Find P and such that A = P P 1 , where
0
0 2
the columns of P are orthonormal and is a diagonal matrix.
Solution:
S
ECE 490: Introduction to Optimization
Spring 2014
Problem Set III
R. Srikant
Quiz: Feb. 13
1.
Suppose
2
M I
f (x) M I x,
where M < is a scalar and I is the identity matrix. Prove that
f (x)
f (y) M y x .
You can use the following facts from linear algeb
ECE 490: Introduction to Optimization
Spring 2014
Problem Set VII
R. Srikant
Quiz: Mar. 20
1. a) (LY) Consider the problem
min
2x2 + 2x1 x2 + x2 10x1 10x2
1
2
s.t.
x2 + x2 5
1
2
3x1 + x2 6.
Find x and that satisfy the rst-order KKT conditions.
b) Is x the
ECE 490: Introduction to Optimization
Spring 2014
Problem Set VIII
R. Srikant
Quiz: Apr. 10
1. Let x be the unique min of the constrained optimization problem
min f (x) .
xS
Suppose there exists a bounded set C such that xk C, k. Here, xk is the solution