Tensor decomposition for Independent Component Analysis
Through implicit cumulant tensor manipulation
P. Denarié (TU Delft - Mechanical Engineering)
K. Batselier – Mentor (TU Delft - Team Kim Batselier)
Borbála Hunyadi – Graduation committee member (TU Delft - Signal Processing Systems)
More Info
expand_more
Github repository containing implementations and experiment code.
https://github.com/padenarie/Independent-component-analysis-through-implicit-cumulant-tensor-decomposition.gitOther than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Blind Source Separation (BSS), the separation of latent source components from observed mixtures, is relevant to many fields of expertise such as neuro-imaging, economics and machine learning. Reliable estimates of the sources can be obtained through diagonalization of the cumulant tensor, i.e., a fourth-order symmetric multi-linear array containing the cross-kurtosis values of observed mixtures. The downside of such diagonalization methods is that they scale quartically with the increase of the amount of source components to estimate due to the tensor’s quartic size. Tensor decomposition can simultaneously diagonalize the cumulant tensor and address its size. However, it does not resolve the scalability issue due to the restriction of having to first explicitly compute the tensor.
It is studied how decomposing the cumulant tensor in implicit fashion can be used to solve the BSS problem while simultaneously addressing its scalablity issue. A class of implicit cumulant tensor decomposition algorithms is derived which scale more favorably than their explicit counterparts in terms of either computational cost, storage cost or both. Firstly, a novel QR-Tensor algorithm (QRT) is introduced which allows for the simultaneous diagonalization of a tensor’s outer-slices. It is theoretically shown how an implicit version of the QRT algorithm can be used to solve the BSS problem at a linearly scaling computational cost. Secondly, a fixed-point Canonical Polyadic Decomposition (CPD) iteration method is presented. It reduces the computational complexity from a quartic dependence to a linear dependence on the amount of signals to estimate. The source estimation performance of the devised implicit decomposition methods is compared to that of the state-of-the-art FastICA for an artificial linear BSS problem.
Results show that both fixed-point CPD and QRT are superior to FastICA when it comes to the computation time needed to reach convergence, while producing estimated sources of similar quality. It is shown that when the amount of sources to estimate is increased both QRT and FastICA struggle to converge. In contrast, the fixed-point CPD method converges within a consistent amount of iterations, suggesting a method more suitable for the estimation of a large amount of sources.