Combining Convolution Neural Network and Bidirectional Gated Recurrent Unit for Sentence Semantic Classification

Many keywords in a sentence that represents the semantic propensity of the sentence. These words can exist anywhere in the sentence, which poses a great challenge to sentence semantic classification. The current sentence semantic classification methods usually tackle this problem by the use of attention mechanism, and most of them utilize softmax function to calculate each word's weight. According to the observation that a word with higher score carries more valuable information in sentence modeling, this paper presents a novel low-complexity model termed as CNN-BiGRU by integrating both convolution neural network (CNN) and bidirectional gated recurrent unit (BiGRU). Both the contextual representations and the semantic distribution are obtained through BiGRU, and the latter is constrained to a Gaussian distribution. In addition, the proposed model utilizes a shallow word-level CNN to obtain intermediate representations, and the score of each word is denoted as the Euclidean distance between the intermediate representations and the semantic distribution. Then, the final representations are obtained by the combination of the contextual representations and the score of each word, and thus, the model learns a compact code for sentence sentiment classification and can be trained end-to-end with limited hyper-parameters. In conclusion, the proposed model is able to focus both the keywords and the underlying semantics of the words. Comprehensive experiments are conducted on seven benchmarks. Compared with the state-of-the-art models, our model has excellent performance.