This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: 1030 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 53, NO. 3, MARCH 2007 Mixing and Non-Mixing Local Minima of the Entropy Contrast for Blind Source Separation Frdric Vrins , Student Member, IEEE , Dinh-Tuan Pham , Member, IEEE , and Michel Verleysen , Senior Member, IEEE Abstract In this paper, both non-mixing and mixing local minima of the entropy are analyzed from the viewpoint of blind source separation (BSS); they correspond respectively to accept- able and spurious solutions of the BSS problem. The contribution of this work is twofold. First, a Taylor development is used to show that the exact output entropy cost function has a non-mixing min- imum when this output is proportional to any of the non-Gaussian sources, and not only when the output is proportional to the lowest entropic source. Second, in order to prove that mixing entropy minima exist when the source densities are strongly multimodal, an entropy approximator is proposed. The latter has the major advantage that an error bound can be provided. Even if this approximator (and the associated bound) is used here in the BSS context, it can be applied for estimating the entropy of any random variable with multimodal density. Index Terms Blind source separation (BSS), entropy estima- tion, independent component analysis, mixture distribution, multi- modal densities. I. INTRODUCTION B LIND source separation (BSS) aims at recovering a vector of independent sources from observed mixtures . In this paper, we assume that and , where is the-by- mixing matrix. The sources can be recovered by finding an unmixing matrix such that is non-mixing (i.e., with one nonzero entry per row and per column). Such matrices can be found by minimizing an ad hoc cost function (see , the books , and references therein). In practice, the minimum of these criteria is reached by adap- tive methods such as gradient descent. Therefore, one has to pay attention to the solutions corresponding to these minima. In most cases, the global minimum is a solution of the BSS problem. By contrast, the possible local minima can either cor- respond to a desired solution (referred as non-mixing minima) or spurious solution (referred as mixing minima) of the problem. For example, the optimization algorithm could be trapped in minima that do not correspond to an acceptable solution of the Manuscript received December 26, 2005; revised October 11, 2006. F. Vrins and M. Verleysen are with the UCL Machine Learning Group, Universit catholique de Louvain, 1348 Louvain-la-Neuve, Belgium (e-mail: email@example.com; firstname.lastname@example.org). D.-T. Pham is with the Laboratoire de Modlisation et Calcul, CNRS, BP 53, 38041 Grenoble, France (e-mail: Dinh-Tuan.Pham@imag.fr)....
View Full Document
- Fall '10