HW5 solns.pdf - Homework 5: Solutions ORIE 4520:...

ORIE 4520: Stochastics at Scale
Fall 2015
Homework 5: Solutions
Sid Banerjee ( [email protected] )
Problem 1: (LSH for Angular Similarity)
For any vectors x, y ∈ R
d
, the angular distance is the angle (in radians) between the two vectors
– formally, d
θ
( x, y ) = cos
- 1
x.y
|| x ||
2
|| y ||
2
(where cos
- 1
( · ) returns the principle angle, i.e., angles in
[0 , π ]). The (normalized) angular similarity is given by s
θ
( x, y ) = 1 - d
θ
( x, y ) /π .
We now want to construct a LSH for the angular similarity metric. Consider the following
family of hash functions: we first choose a random unit vector σ (i.e., σ ∈ R
d
with || σ ||
2
= 1), and
for any vector x , define h
σ
( x ) = sgn ( x.σ ) (i.e., the sign of the dot product of x and σ ). Argue that
for any x, y ∈ R
d
, we have:
P [ h
σ
( x ) = h
σ
( y )] = s
θ
( x, y )
Hint: For any pair x and y in R
d
, there is a unique plane passing through the origin containing x
and y – convince yourself that d
θ
( x, y ) is precisely the angle between x and y in this plane. Also,
given any vector σ , its dot product with x and y only depends on the projection of σ on this plane.
Now what can you say about the signs of the dot products of x and y with a random unit vector?

ORIE 4520: Stochastics at Scale
Fall 2015
Homework 5: Solutions
Sid Banerjee ( [email protected] )
projection onto the plane of x and y is above the dashed line in Fig. (1), then σ.x is positive, while
σ.y is negative. The normal vector σ instead might extend in the opposite direction, below the
dashed line. In that case σ.x is negative and σ.y is positive, but the signs are still different.
On the other hand, the randomly chosen vector σ could be normal to a hyperplane like the
dotted line in Fig. (1). In that case, both σ.x and σ.y have the same sign. If the projection of σ
extends to the right, then both dot products are positive, while if σ extends to the left, then both
are negative.
What is the probability that the randomly chosen vector is normal to a hyperplane that looks
like the dashed line rather than the dotted line? All angles for the line that is the intersection of
the random hyperplane and the plane of x and y are equally likely. Thus, the hyperplane will look
like the dashed line with probability θ/π and will look like the dotted line otherwise.