Text Classification and Transfer Learning Based on Character-Level Deep Convolutional Neural Networks

Temporal (one-dimensional) Convolutional Neural Network (Temporal CNN, ConvNet) is an emergent technology for text understanding. The input for the ConvNets could be either a sequence of words or a sequence of characters. In the latter case there are no needs for natural language processing. Past studies showed that the character-level ConvNets worked well for text classification in English and romanized Chinese corpus. In this article we apply the character-level ConvNets to Japanese corpus. We confirmed that meaningful representations are extracted by the ConvNets in English corpus and Japanese corpus. We attempt to reuse the meaningful representations that are learned in the ConvNets from a large-scale dataset in the form of transfer learning. As for the application to the news categorization and the sentiment analysis tasks in Japanese corpus, the ConvNets outperformed N-gram-based classifiers. In addition, our ConvNets transfer learning frameworks worked well for a task which is similar to one used for pre-training.