4 Independent component analysis

Independent Component Analysis (ICA) is a useful extension of PCA that was developed in context with source or signal separation applications (Oja et al. 1996): instead of requiring that the coefficients of a linear expansion of data vectors are uncorrelated, in ICA they must be mutually independent or as independent as possible. This implies that second order moments are not sufficient, but higher order statistics are needed in determining ICA. This provides a more meaningful representation of data than PCA. In current ICA methods based on PCA neural networks, the following data model is usually assumed. The L-dimensional k-th data vector $\vec{x}_k$ is of the form (Oja et al. 1996):

 

$$\vec{x}_{k}={\bf A}{\rm s}_{k}+\vec{n}_{k}=\sum_{i=1}^{M}s_{k}(i)\vec{a}(i)+ \vec{n}_{k}$$ (9)

where in the M-vector $\vec{s}_{k}=[s_{k}(1),~\ldots ,s_{k}(M)]^{\rm T}$, s_k(i) denotes the i-th independent component (source signal) at time k, ${\bf A}=[\vec{a}(1),~\ldots ,\vec{a}(M)]$ is a $L\times M$ mixing matrix whose columns $\vec{a}(i)$ are the basis vectors of ICA, and $\vec{n}_{k}$ denotes noise.

The source separation problem is now to find an $M\times L$ separating matrix ${\bf B}$ so that the M-vector $\vec{y}_{k}={\bf B}\vec{x}_{k}$ is an estimate $\vec{y}_{k}=\vec{\hat{s}}_{k}$ of the original independent source signal (Oja et al. 1996).

4.1 Whitening

Whitening is a linear transformation ${\bf A}$ such that, given a matrix ${\bf C}$, we have ${\bf ACA}^{\rm T}={\bf D}$ where ${\bf D}$ is a diagonal matrix with positive elements (Kay 1988; Marple 1987).

Several separation algorithms utilise the fact that if the data vectors $\vec{x}_k$ are first pre-processed by whitening them (i.e. $E(x_{k}x_{k}^{\rm T})=I$ with E(.) denoting the expectation), then the separating matrix ${\bf B}$ becomes orthogonal (${\bf BB}^{\rm T}={\bf I}$, see (Oja et al. 1996)).

Approximating contrast functions which are maximised for a separating matrix have been introduced because the involved probability densities are unknown (Oja et al. 1996).

It can be shown that, for prewhitened input vectors, the simpler contrast function given by the sum of kurtoses is maximised by a separating matrix ${\bf B}$ (Oja et al. 1996).

However, we found that in our experiments the whitening was not as good as we expected, because the estimated frequencies calculated for prewhitened signals with the neural estimator (n.e.) were not too much accurate.

In fact we can pre-elaborate the signal, whitening it, and then we can apply the n.e. Otherwise we can apply the whitening and separate the signal in independent components with the nonlinear neural algorithm of Eq. (8) and then apply the n.e. to each of these components and estimate the single frequencies separately.

The first method gives comparable or worse results than n.e. without whitening. The second one gives worse results and is very expensive. When we used the whitening in our n.e. the results were worse and more time consuming than the ones obtained using the standard n.e. (i.e. without whitening the signal). Experimental results are given in the following sections. For these reasons whitening is not a suitable technique to improve our n.e..