An Overcomplete ICA Algorithm by InfoMax and InfoMin

It is known that the number of the edge detectors significantly e xceeds that of input signals in the visual system of the brains. This phenomenon has been often regarded as overcomplete indepenent component analysis (ICA) and some generative models have been proposed. Though the models are effective, they need to assume some ad-hoc prior probabilistic models. Recently, the InfoMin principle was proposed as a comprehensive framework with minimal prior assumptions for explaining the information processing in the brains and its usefulness has been verified in the classic non-overcomplete cases. In this paper, we propose a new ICA contrast function for overcomplete cases, which is deductively derived from the InfoMin and InfoMax principles without any priior models. Besides, we construct an efficient fixed-point algorithm for optimizing it by an approximate Newton's method. Numerical experiments verify the effectiveness of the proposed method.