MIT6_079F09_lec11

MIT6_079F09_lec11 - ℓ 1-norm Methods for...

Info iconThis preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ℓ 1-norm Methods for Convex-Cardinality Problems • problems involving cardinality • the ℓ 1-norm heuristic • convex relaxation and convex envelope interpretations • examples • recent results Prof. S. Boyd, EE364b, Stanford University ℓ 1-norm heuristics for cardinality problems • cardinality problems arise often, but are hard to solve exactly • a simple heuristic, that relies on ℓ 1-norm, seems to work well • used for many years, in many fields – sparse design – LASSO, robust estimation in statistics – support vector machine (SVM) in machine learning – total variation reconstruction in signal processing, geophysics – compressed sensing • new theoretical results guarantee the method works, at least for a few problems Prof. S. Boyd, EE364b, Stanford University 1 Cardinality • the cardinality of x ∈ R n , denoted card ( x ) , is the number of nonzero components of x 0 x = 0 • card is separable; for scalar x , card ( x ) = 1 x = 0 • card is quasiconcave on R n (but not R n ) since + card ( x + y ) ≥ min { card ( x ) , card ( y ) } holds for x, y 0 • but otherwise has no convexity properties • arises in many problems Prof. S. Boyd, EE364b, Stanford University 2 General convex-cardinality problems a convex-cardinality problem is one that would be convex, except for appearance of card in objective or constraints examples (with C , f convex): • convex minimum cardinality problem: minimize card ( x ) subject to x ∈ C • convex problem with cardinality constraint: minimize f ( x ) subject to x ∈ C , card ( x ) ≤ k Prof. S. Boyd, EE364b, Stanford University 3 Solving convex-cardinality problems convex-cardinality problem with x ∈ R n • if we fix the sparsity pattern of x ( i.e. , which entries are zero/nonzero) we get a convex problem • by solving 2 n convex problems associated with all possible sparsity patterns, we can solve convex-cardinality problem (possibly practical for n ≤ 10 ; not practical for n > 15 or so . . . ) • general convex-cardinality problem is (NP-) hard • can solve globally by branch-and-bound – can work for particular problem instances (with some luck) – in worst case reduces to checking all (or many of) 2 n sparsity patterns Prof. S. Boyd, EE364b, Stanford University 4 Boolean LP as convex-cardinality problem • Boolean LP: minimize c T x subject to Ax b, x i ∈ { , 1 } includes many famous (hard) problems, e.g. , 3-SAT, traveling salesman • can be expressed as minimize c T x subject to Ax b, card ( x ) + card (1 − x ) ≤ n since card ( x ) + card (1 − x ) ≤ n ⇐⇒ x i ∈ { , 1 } • conclusion: general convex-cardinality problem is hard Prof. S. Boyd, EE364b, Stanford University 5 Sparse design minimize card ( x ) subject to x ∈ C • find sparsest design vector x that satisfies a set of specifications • zero values of x simplify design, or correspond to components that aren’t even needed • examples: – FIR filter design (zero coefficients reduce required hardware)...
View Full Document

This note was uploaded on 01/31/2012 for the course ECE 202 taught by Professor Boyde during the Fall '06 term at Goldsmiths.

Page1 / 32

MIT6_079F09_lec11 - ℓ 1-norm Methods for...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online