Improving convolutional neural network for text classification by recursive data pruning

被引:27
|
作者
Li, Qi [2 ]
Li, Pengfei [1 ]
Mao, Kezhi [1 ]
Lo, Edmond Yat-Man [2 ,3 ]
机构
[1] Nanyang Technol Univ, Sch Elect & Elect Engn, Singapore 639798, Singapore
[2] Nanyang Technol Univ, Inst Catastrophe Risk Management, Interdisciplinary Grad Programme, Singapore 639798, Singapore
[3] Nanyang Technol Univ, Sch Civil & Environm Engn, Singapore 639798, Singapore
关键词
Data pruning; Convolutional neural network; Text classification; SENTIMENT ANALYSIS;
D O I
10.1016/j.neucom.2020.07.049
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In spite of the state-of-the-art performance of deep neural networks, shallow neural networks are still the choice in applications with limited computing and memory resources. Convolutional neural network (CNN), in particular the one-convolutional-layer CNN, is a widely-used shallow neural network in natural language processing tasks such as text classification. However, it was found that CNNs may misfit to task-irrelevant words in dataset, which in turn leads to unsatisfactory performance. To alleviate this problem, attention mechanism can be integrated into CNN, but this takes up the limited resources. In this paper, we propose to address the misfitting problem from a novel angle - pruning task-irrelevant words from the dataset. The proposed method evaluates the performance of each convolutional filter based on its discriminative power of the feature generated at the pooling layer, and prunes words captured by the poorly-performed filters. Experiment results show that our proposed model significantly outperforms the CNN baseline model. Moreover, our proposed model produces performance similar to or better than the benchmark models (attention integrated CNNs) while demanding less parameters and FLOPs, and is therefore a choice model for resource limited scenarios, such as mobile applications. (C) 2020 Elsevier B.V. All rights reserved.
引用
收藏
页码:143 / 152
页数:10
相关论文
共 50 条
  • [1] Transformable Convolutional Neural Network for Text Classification
    Xiao, Liqiang
    Zhang, Honglun
    Chen, Wenqing
    Wang, Yongkun
    Jin, Yaohui
    PROCEEDINGS OF THE TWENTY-SEVENTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2018, : 4496 - 4502
  • [2] Fault Text Classification Based on Convolutional Neural Network
    Wang, Lixia
    Zhang, Botao
    2020 IEEE 7TH INTERNATIONAL CONFERENCE ON INDUSTRIAL ENGINEERING AND APPLICATIONS (ICIEA 2020), 2020, : 937 - 941
  • [3] Application of Improved Convolutional Neural Network in Text Classification
    Ronghui, Liu
    Xinhong, Wei
    IAENG International Journal of Computer Science, 2022, 49 (03)
  • [4] Application of Convexified Convolutional Neural Network in Text Classification
    Bian, Yuanchong
    Li, Chang
    Wang, Bincheng
    Zhang, Xingjian
    2021 IEEE INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS AND COMPUTER ENGINEERING (ICCECE), 2021, : 296 - 300
  • [5] Combining Convolutional Neural Network With Recursive Neural Network for Blood Cell Image Classification
    Liang, Gaobo
    Hong, Huichao
    Xie, Weifang
    Zheng, Lixin
    IEEE ACCESS, 2018, 6 : 36188 - 36197
  • [6] Improving Short Text Classification Using Fast Semantic Expansion on Multichannel Convolutional Neural Network
    Sotthisopha, Natthapat
    Vateekul, Peerapon
    2018 19TH IEEE/ACIS INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING, ARTIFICIAL INTELLIGENCE, NETWORKING AND PARALLEL/DISTRIBUTED COMPUTING (SNPD), 2018, : 182 - 187
  • [7] Improving Text Classification in Agricultural Expert Systems with a Bidirectional Encoder Recurrent Convolutional Neural Network
    Guo, Xiaojuan
    Wang, Jianping
    Gao, Guohong
    Li, Li
    Zhou, Junming
    Li, Yancui
    ELECTRONICS, 2024, 13 (20)
  • [8] Variational Convolutional Neural Network Pruning
    Zhao, Chenglong
    Ni, Bingbing
    Zhang, Jian
    Zhao, Qiwei
    Zhang, Wenjun
    Tian, Qi
    2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 2775 - 2784
  • [9] Convolutional Neural Network Pruning: A Survey
    Xu, Sheng
    Huang, Anran
    Chen, Lei
    Zhang, Baochang
    PROCEEDINGS OF THE 39TH CHINESE CONTROL CONFERENCE, 2020, : 7458 - 7463
  • [10] APPLICATION OF CONVOLUTIONAL NEURAL NETWORK (CNN) IN MICROBLOG TEXT CLASSIFICATION
    Wang, Xiaoming
    Li, Jianping
    Liu, Yifei
    2018 15TH INTERNATIONAL COMPUTER CONFERENCE ON WAVELET ACTIVE MEDIA TECHNOLOGY AND INFORMATION PROCESSING (ICCWAMTIP), 2018, : 127 - 130