Speech signals separation: A new approach exploiting the coherence of audio and visual speech

In this paper, we present a new approach to the source separation problem in the case of multiple speech signals. The method is based on the use of automatic lipreading: the objective is to extract an acoustic speech signal from other acoustic signals by exploiting its coherence with the speaker's lip movements. For this aim, a statistical model is used to quantify this coherence. The results, while very preliminary, are encouraging. They show that this method can achieve a good separation of a speech source in the case of simple W additive mixtures. Moreover, it presents some Interesting complementarity with traditional pure audio techniques.