This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: ConvexOptimizationII-Lecture15 Instructor (Stephen Boyd) :All right, I think this means we are on. There is no good way in this room to know if you are – when the lecture starts. Okay, well, we are down to a skeleton crew here, mostly because it’s too hot outside. So we’ll continue with L_1 methods today. So last time we saw the basic idea. The most – the simplest idea is this. If you want to minimize the cardinality of X, find the sparsest vector X that’s in a convex set, the simplest heuristic – and actually, today, we’ll see lots of variations on it that are more sophisticated. But the simplest one, by far, is simply to minimize the one norm of X subject to X and Z. By the way, all of the thousands of people working on L_1, this is all they know. So the things we are going to talk about today, basically most people don’t know. All right. We looked at that. Last time we looked at polishing, and now I want to interpret this – I want to justify this L_1 norm heuristic. So here is one. We can turn this – we can interpret this as a relaxation of – we can make this a relaxation of a Boolean convex problem. So what we do is this. I am going to rewrite this cardinality problem this way. I am going to introduce some Boolean variables Z. And these are basically indicators that tell you whether or not each component is either zero or nonzero. And I’ll enforce it this way. I’ll say that the absolute value is XI is less than RZI. Now R is some number that bounds, for example – like it could be just basically a bounding box for C, or it can be naturally part of the constraints. It really doesn’t matter. The point is that any feasible point here has an infinity norm less than R. If we do this like this, we end up with this problem. This problem is a Boolean convex. And what that means is that it is – everything is convex, and the variables, that’s X and Z, except for one minor problem, and that is that these are 01. Okay? So this is a Boolean convex problem. And it’s absolutely equivalent to this one. It is just as hard, of course. So we are going to do the standard relaxation is if you have a 01 – 0, 1 variable, we’ll change it into a left bracket 0,1 right bracket variable. And that means that it’s a continuous variable. This is a relaxation. And here, we have simply – we have actually worked out, this is simply – well, it’s obvious enough, but this is simply the convex hull of the Boolean points here. Now if you stare at this long enough, you realize something. You have seen this before. This is precisely the linear program that defines – this is exactly the linear program that defines the L_1 norm. So here, for example, this is at norm X is – it’s an upper bound on – ZI is an upper bound on one over RXI. And so, in fact, this problem is absolutely the same as this one. And so now you see what you have. That Boolean problem is equivalent to this. By the way, this tells you something. It says that when you solve that L_1 problem, not only do you have tells you something....
View Full Document
- Fall '09
- Maximum likelihood, Convex set, Convex function, Stephen Boyd