Adversarial Dictionary Learning for Monaural Speech Enhancement

In this paper, we propose an adversarial dictionary learning method to train a speaker independent speech dictionary and a universal noise dictionary for improving the generality of the dictionary learning based speech enhancement system. In the learning stage, two discriminators are employed separately to identify the components in speech and noise which are highly correlated with each other. The residuals in the speech and noise magnitude spectral matrices are then utilized to train the speech and noise dictionaries via the alternating direction method of multiplier algorithm, which can effectively reduce the mutual coherence between speech and noise. In the enhancement stage, a new optimization technique is proposed for enhancing the speech based on the low-rank decomposition and sparse coding. Experimental results show that our proposed method achieves better performance in improving the speech quality and intelligibility than the reference methods in terms of three objective performance evaluation measures.