This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: ℓ 1norm Methods for ConvexCardinality Problems • problems involving cardinality • the ℓ 1norm heuristic • convex relaxation and convex envelope interpretations • examples • recent results Prof. S. Boyd, EE364b, Stanford University ℓ 1norm heuristics for cardinality problems • cardinality problems arise often, but are hard to solve exactly • a simple heuristic, that relies on ℓ 1norm, seems to work well • used for many years, in many fields – sparse design – LASSO, robust estimation in statistics – support vector machine (SVM) in machine learning – total variation reconstruction in signal processing, geophysics – compressed sensing • new theoretical results guarantee the method works, at least for a few problems Prof. S. Boyd, EE364b, Stanford University 1 Cardinality • the cardinality of x ∈ R n , denoted card ( x ) , is the number of nonzero components of x 0 x = 0 • card is separable; for scalar x , card ( x ) = 1 x = 0 • card is quasiconcave on R n (but not R n ) since + card ( x + y ) ≥ min { card ( x ) , card ( y ) } holds for x, y 0 • but otherwise has no convexity properties • arises in many problems Prof. S. Boyd, EE364b, Stanford University 2 General convexcardinality problems a convexcardinality problem is one that would be convex, except for appearance of card in objective or constraints examples (with C , f convex): • convex minimum cardinality problem: minimize card ( x ) subject to x ∈ C • convex problem with cardinality constraint: minimize f ( x ) subject to x ∈ C , card ( x ) ≤ k Prof. S. Boyd, EE364b, Stanford University 3 Solving convexcardinality problems convexcardinality problem with x ∈ R n • if we fix the sparsity pattern of x ( i.e. , which entries are zero/nonzero) we get a convex problem • by solving 2 n convex problems associated with all possible sparsity patterns, we can solve convexcardinality problem (possibly practical for n ≤ 10 ; not practical for n > 15 or so . . . ) • general convexcardinality problem is (NP) hard • can solve globally by branchandbound – can work for particular problem instances (with some luck) – in worst case reduces to checking all (or many of) 2 n sparsity patterns Prof. S. Boyd, EE364b, Stanford University 4 Boolean LP as convexcardinality problem • Boolean LP: minimize c T x subject to Ax b, x i ∈ { , 1 } includes many famous (hard) problems, e.g. , 3SAT, traveling salesman • can be expressed as minimize c T x subject to Ax b, card ( x ) + card (1 − x ) ≤ n since card ( x ) + card (1 − x ) ≤ n ⇐⇒ x i ∈ { , 1 } • conclusion: general convexcardinality problem is hard Prof. S. Boyd, EE364b, Stanford University 5 Sparse design minimize card ( x ) subject to x ∈ C • find sparsest design vector x that satisfies a set of specifications • zero values of x simplify design, or correspond to components that aren’t even needed • examples: – FIR filter design (zero coeﬃcients reduce required hardware)...
View
Full
Document
This note was uploaded on 01/31/2012 for the course ECE 202 taught by Professor Boyde during the Fall '06 term at Goldsmiths.
 Fall '06
 boyde

Click to edit the document details