Tightly-coupled Convolutional Neural Network with Spatial-temporal Memory for Text Classification

Although several traditional models like bag of words (BOW), n-grams, and their variants of TFIDF exhibit high performance in the field of text classification, neural network methods such as LSTM, GRU and convolutional neural network (CNN) are recently attracting increasing attention. Considering that CNN has surprising capabilities of extracting hierarchical features, combination of LSTM/GRU with CNN seems to be quite reasonable for semantic representation and sequence analysis. On the other hand, it is also a promising subject to enable CNN to have memory embeddings and/or recurrent pathway. In this paper, we propose a novel tightly-coupled convolutional neural network with spatial-temporal memory (TCNN-SM). It comprises feature-representation and memory functional columns. Feature-representation functional column in our TCNN-SM actually performs hierarchical feature extraction as regular CNN does while memory functional column retains memories of different granularity and fulfills selective memory for historical information. In order to validate effectiveness and efficiency of the proposed TCNN-SM, we conduct extensive experiments on AG's News public dataset. The experimental results show that our new TCNN-SM achieves 7.99% test error, which has the best performance among other existing deep learning methods and is very close to state of the art results yielded using classical n-grams algorithm.