A Multi-scale Convolutional Attention Based GRU Network for Text Classification

Neural network models have been widely used in natural language processing (NLP). Recurrent neural network (RNNs) has proved to be a powerful sequence model. Gated recurrent unit (GRU) is one kind of RNNs which has achieved excellent performance in NLP. Nevertheless, because of the sparsity and high dimensionality of text data, there are some difficulties in complex semantic representations. To solve these problems, a novel and efficient method is proposed in this paper for text classification. The proposed model is called multi-scale convolutional attention based GRU network (MCA-GRU). In MCA-GRU, one-dimension convolutions with dense connections extract attention signals from text sequences. Then the attention signals are combined with features of GRU network. MCA-GRU is able to capture the local feature of phrases and sequence information. Experimental verifications are conducted on five text classification datasets. The results clearly show that the proposed model MCA-GRU approach achieves equivalent or even superior performance than other state-of-the-art text classification methods.