This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Gradient Descent with Sparsification: An iterative algorithm for sparse recovery with restricted isometry property Rahul Garg [email protected] Rohit Khandekar [email protected] IBM T. J. Watson Research Center, 1101 Kitchawan Road, Route 134, Yorktown Heights, NY 10598 Abstract We present an algorithm for finding an s sparse vector x that minimizes the square error k y Φ x k 2 where Φ satisfies the re stricted isometry property (RIP), with iso metric constant δ 2 s < 1 / 3. Our algorithm, called GraDeS (Gradient Descent with Spar sification) iteratively updates x as: x ← H s x + 1 γ · Φ > ( y Φ x ) where γ > 1 and H s sets all but s largest magnitude coordinates to zero. GraDeS con verges to the correct solution in constant number of iterations. The condition δ 2 s < 1 / 3 is most general for which a nearlinear time algorithm is known. In comparison, the best condition under which a polynomial time algorithm is known, is δ 2 s < √ 2 1. Our Matlab implementation of GraDeS out performs previously proposed algorithms like Subspace Pursuit, StOMP, OMP, and Lasso by an order of magnitude. Curiously, our experiments also uncovered cases where L1regularized regression (Lasso) fails but GraDeS finds the correct solution. 1. Introduction Finding a sparse solution to a system of linear equa tions has been an important problem in multiple do mains such as model selection in statistics and ma chine learning (Golub & Loan, 1996; Efron et al., 2004; Wainwright et al., 2006; Ranzato et al., 2007), sparse principal component analysis (Zou et al., 2006), image Appearing in Proceedings of the 26 th International Confer ence on Machine Learning , Montreal, Canada, 2009. Copy right 2009 by the author(s)/owner(s). deconvolution and denoising (Figueiredo & Nowak, 2005) and compressed sensing (Cand` es & Wakin, 2008). The recent results in the area of compressed sensing, especially those relating to the properties of random matrices (Cand` es & Tao, 2006; Cand` es et al., 2006) has exploded the interest in this area which is finding applications in diverse domains such as coding and information theory, signal processing, artificial in telligence, and imaging. Due to these developments, efficient algorithms to find sparse solutions are increas ingly becoming very important. Consider a system of linear equations of the form y = Φ x (1) where y ∈ < m is an mdimensional vector of “mea surements”, x ∈ < n is the unknown signal to be recon structed and Φ ∈ < m × n is the measurement matrix. The signal x is represented in a suitable (possibly over complete) basis and is assumed to be “ ssparse” (i.e. at most s out of n components in x are nonzero). The sparse reconstruction problem is min ˜ x ∈< n  ˜ x  subject to y = Φ˜ x (2) where  ˜ x  represents the number of nonzero entries in ˜ x . This problem is not only NPhard (Natarajan, 1995), but also hard to approximate within a factor O (2 log 1 ( m ) ) of the optimal solution (Neylon, 2006).) of the optimal solution (Neylon, 2006)....
View
Full Document
 Spring '11
 Li
 Linear Programming, Signal Processing, Gradient descent, subspace pursuit

Click to edit the document details