Speech Separation and Recognition Using CASA Segmentation and Language-Based Grouping

We consider a monaural speech recognition problem in the case of multi-talker environment and difficult non-stationary noises. We propose a new method of computational auditory scene analysis (CASA) that uses a language model along with acoustic continuity for speech separation. Unlike previous works, our algorithm does not depend on a fixed set of speakers, so it could be used in a general-purpose speech recognition system. The algorithm works in two stages. First, it produces time-frequency signal segmentation. Then, a grouping stage composes segments into streams, with each stream corresponding to either speech or noise. In our approach, text recognition and separation are parts of a single process. Our experiments show 17% WER improvement over the baseline for a 0 dB environment.