Implementation of Convolutional Neural Network for Speech Recognition

Thanks to the contribution of Professor Hinton, a pre-trained context-dependent hybrid deep neural network (DNN) has achieved significant performance gain in many automatic speech recognition tasks. And convolutional neural network (CNN) is another type of neural network which has been successfully applied in many image processing tasks. Due to its unique structure (local filtering and pooling), CNN has some advantage over DNN such as smaller model size and translation invariance. In this paper, we implement convolutional neural network for speech recognition. We first apply CNN on the TIMIT database using the method proposed in [1]. Although it shows gains compared with DNN, it is too slow to train the net. Then we proposed a method to accelerate the training. Next we apply the system for the Mandarin task which have more data and is much noisier than TIMIT. The experiments indicate that it also works and we get the best results with two convolutional layers.