This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: 626 IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999 Fast and Robust Fixed-Point Algorithms for Independent Component Analysis Aapo Hyvarinen Abstract Independent component analysis (ICA) is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible. In this paper, we use a combination of two different approaches for linear ICA: Comons information- theoretic approach and the projection pursuit approach. Using maximum entropy approximations of differential entropy, we introduce a family of new contrast (objective) functions for ICA. These contrast functions enable both the estimation of the whole decomposition by minimizing mutual information, and estima- tion of individual independent components as projection pursuit directions. The statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model, and it is shown how to choose contrast functions that are robust and/or of minimum variance. Finally, we introduce simple xed-point algorithms for practical optimization of the contrast functions. These algorithms optimize the contrast functions very fast and reliably. I. INTRODUCTION A CENTRAL problem in neural-network research, as well as in statistics and signal processing, is nding a suitable representation or transformation of the data. For computational and conceptual simplicity, the representation is often sought as a linear transformation of the original data. Let us denote by a zero-mean-dimensional random variable that can be observed, and by its-dimensional transform. Then the problem is to determine a constant (weight) matrix so that the linear transformation of the observed variables (1) has some suitable properties. Several principles and methods have been developed to nd such a linear representation, including principal component analysis , factor analysis , projection pursuit , , independent component analysis , etc. The transformation may be dened using such criteria as optimal dimension reduction, statistical inter- estingness of the resulting components , simplicity of the transformation, or other criteria, including application-oriented ones. We treat in this paper the problem of estimating the trans- formation given by (linear) independent component analysis (ICA) , . As the name implies, the basic goal in determining the transformation is to nd a representation in which the transformed components are statistically as Manuscript received November 20, 1997; revised November 19, 1998 and January 29, 1999....
View Full Document