Independent Component Analysis: Principles and Practice
 
Chapter summaries
 
 
 
          
 
 
  
Introduction
Stephen Roberts and Richard Everson
Chapter 1 offers an introduction to independent component analysis. This chapter aims to give the reader an accessible way into the techniques, issues and jargon of ICA. The field is an extensive one and we have attempted to keep to ideas which we regard as instructive in the key issues of ICA, rather than give a complete description of every sub-method and algorithm modification.
Fast ICA by a fixed-point algorithm that maximizes non-Gaussianity
Aapo Hyvärinen
Chapter 2 details one of the most popular approaches to ICA based on polynomial approximations to the mutual information, that of Fast ICA. Hyvärinen details in this chapter the theoretical development of the approximations, their justification and the rapid fixed-point algorithm by which the sources are recovered. Background material on the relationship between learning algorithms is also presented along with results on a number of datasets.
ICA, graphical models and variational methods
Hagai Attias
Chapter 3 pitches ICA into the important context of graphical models, whereby the relationships between model parameters are represented by a directed acyclic graph. Attias, in this chapter, considers flexible ICA methods (sometimes known as Independent Factor Analysis, IFA) in which the source densities are modelled by mixtures of Gaussians and an explicit additive (sensor) noise term exists. Inference is performed using a variational learning approach (see also Chapter 8).
Nonlinear ICA
Juha Karhunen
Chapter 4 extends ICA from a general linear model of source mixing to the nonlinear case. Karhunen explores in detail the issues involved with forming learning paradigms for such nonlinear ICA and develops promising algorithms to deal with nonlinear mixing. The chapter is illustrated with comparative examples.
Separation of non-stationary natural signals
Lucas Parra and Clay Spence
Chapter 5 extends ICA to consider the issue of source non-stationarity. Parra and Spence show how the higher order statistics used to locate independent components arise naturally in non-stationary signals. They examine whether linear mixing is a good model for acoustic signals and natural images. Exploiting the property of non-stationarity they show how good blind separation may be achieved using multiple linear decorrelation.
Separation of non-stationary sources: algorithms and performance
Jean-François Cardoso and Dinh-Tuan Pham
Chapter 6 offers a different perspective on the separation of non-stationary sources. Cardoso and Pham derive an elegant methodology in which non-stationarity may be handled and indeed utilised to aid in the unmixing process.
Blind source separation by sparse decomposition of a signal dictionary
Michael Zibulevsky, Barak Pearlmutter, Pau Bofill and Pavel Kisilev
Chapter 7 considers source separation in the case when the sources are represented by a \textit{sparse} mixture from a signal dictionary (such as wavelet packets). Under these circumstances ICA naturally seeks sources which are as sparse in their representation as possible. This extra information enables Zibulevsky, Pearlmutter, Bofill and Kisilev to obtain impressive results in situations when there are \textit{more} sources than observations.
Ensemble learning for blind source separation
James Miskin and David MacKay
Chapter 8 pitches ICA as a graphical model with densities over variables being inferred using a variational learning framework. As both parameters and hyper-parameters of the model are inferred as part of a single learning strategy, this approach is referred to as ensemble learning. Miskin and MacKay consider mixture of Gaussian source models and show results from model-selection on real-world problems. They also show that a positivity constraint on the hypothesised mixing process gives rise to ICA solutions which are more local in their support.
Image processing methods using ICA mixture models
Te-Won Lee and Michael S. Lewicki
Chapter 9 applies ICA to the domain of image decomposition and processing. By assuming that natural images are generated by a linear combination of independent sources (textures and edges for example) ICA may be used to estimate, given an image, a basis for decomposition. Lee and Lewicki show that this basis has an intuitively appealing form (typically that of local filters) and show how ICA may thence be utilised to perform image denoising.
Latent class and trait models for data classification and visualisation
Mark Girolami
Chapter 10 regards ICA as a general linear transform of the same form as the linear discriminant of pattern classification. Using a nested hierarchy of ICA decompositions of real data, Girolami shows that excellent results may be obtained in difficult unsupervised classification problems. He then proceeds to consider the process of visualisation of high-dimensional data (mapping to a two-dimensional space, for example) as an ICA-like procedure for which learning rules may be obtained.
Particle filters for non-stationary ICA
Richard Everson and Stephen Roberts
Chapter 11 allows a model in which the mixing matrix of a linear ICA model is considered to be non-stationary. This matrix may thence be tracked using a particle filter. The approach is shown to be very effective when tracking non-stationary mixing of temporally uncorrelated sources. Some solutions to the more difficult problem of tracking mixtures of temporally correlated sources are presented.
ICA: model order selection and dynamic source models
William Penny, Stephen Roberts and Richard Everson
Chapter 12 extends the standard ICA model by allowing the source density models to be dynamic rather than static. This is achieved by the use of linear dynamic models within the ICA framework. The authors also consider the important issue of model order selection to determine the most probable number of underlying sources. Results are presented for a variety of datasets.