Combining attention-based bidirectional gated recurrent neural network and two-dimensional convolutional neural network for document-level sentiment classification

被引:56
|
作者
Liu, Fagui [1 ]
Zheng, Jingzhong [1 ]
Zheng, Lailei [1 ]
Chen, Cheng [1 ]
机构
[1] South China Univ Technol, Sch Comp Sci & Engn, Guangzhou 510006, Guangdong, Peoples R China
关键词
Sentiment classification; Attention network; 2DCNN; Bidirectional GRU; DISTRIBUTED REPRESENTATIONS;
D O I
10.1016/j.neucom.2019.09.012
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Neural networks lately have achieved a great success on sentiment classification due to their ability of feature extraction. However, it remains as an enormous challenge to model long texts in document-level sentiment classification as well as to explore semantics between sentences and dependencies between features. Moreover, most existing methods can hardly differentiate the importance of different contents when constructing a document representation. To tackle these problems, we propose a novel neural network model: AttDR-2DCNN, which mainly consists of two parts. The first part utilizes a two-layer compositional bidirectional Gated Recurrent Unit (GRU) to obtain the compositional semantics of the document, where the first layer learns the feature vector of the sentence, and the second layer learns the document matrix representation consisting of two dimensions of the time-step dimension and feature dimension, from the sentence vectors. The second part applies a two-dimensional convolution operation and two-dimensional max pooling to capture more dependencies between sentences features. We further utilize different types of attention mechanism in these two parts to distinguish the importance of words, sentences and features in the document. Experiments are conducted on four publicly available document-level review datasets and the result shows that the proposed model outperforms some existing models. (C) 2019 Elsevier B.V. All rights reserved.
引用
收藏
页码:39 / 50
页数:12
相关论文
共 50 条
  • [41] Attention-Based Recurrent Neural Network for Multicriteria Recommendations
    Bougteb, Yahya
    Frikh, Bouchra
    Ouhbi, Brahim
    Zemmouri, El Moukhtar
    INTELLIGENT SYSTEMS AND APPLICATIONS, VOL 2, INTELLISYS 2023, 2024, 823 : 264 - 274
  • [42] Attention-Based Recurrent Neural Network for Sequence Labeling
    Li, Bofang
    Liu, Tao
    Zhao, Zhe
    Du, Xiaoyong
    WEB AND BIG DATA (APWEB-WAIM 2018), PT I, 2018, 10987 : 340 - 348
  • [43] Attention-based Recurrent Neural Network for Location Recommendation
    Xia, Bin
    Li, Yun
    Li, Qianmu
    Li, Tao
    2017 12TH INTERNATIONAL CONFERENCE ON INTELLIGENT SYSTEMS AND KNOWLEDGE ENGINEERING (IEEE ISKE), 2017,
  • [44] A multi-layer two-dimensional convolutional neural network for sentiment analysis
    Wang, Lin
    Meng, Zuqiang
    Yang, Lina
    INTERNATIONAL JOURNAL OF BIO-INSPIRED COMPUTATION, 2022, 19 (02) : 97 - 107
  • [45] Analyzing sleep thermal comfort with an attention-based gated recurrent unit neural network
    Tang, Jishen
    Li, Jilei
    Wang, Jiang
    Li, Yunhao
    Yang, Yimin
    Song, Zuoting
    Ma, Meirong
    Deng, Bin
    BUILDING AND ENVIRONMENT, 2024, 262
  • [46] Product quality time series prediction with attention-based convolutional recurrent neural network
    Shi, Yiguan
    Chen, Yong
    Zhang, Longjie
    APPLIED INTELLIGENCE, 2024, 54 (21) : 10763 - 10779
  • [47] Sentiment-aware Short Text Classification Based on Convolutional Neural Network and Attention
    Chen, Zeyu
    Tang, Yan
    Zhang, Zuowei
    Zhang, Chengyang
    Wang, Luwei
    2019 IEEE 31ST INTERNATIONAL CONFERENCE ON TOOLS WITH ARTIFICIAL INTELLIGENCE (ICTAI 2019), 2019, : 1172 - 1179
  • [48] Document-Level Text Classification Using Single-Layer Multisize Filters Convolutional Neural Network
    Akhter, Muhammad Pervez
    Jiangbin, Zheng
    Naqvi, Irfan Raza
    Abdelmajeed, Mohammed
    Mehmood, Atif
    Sadiq, Muhammad Tariq
    IEEE ACCESS, 2020, 8 (08): : 42689 - 42707
  • [49] Combination of Convolutional Neural Network and Gated Recurrent Unit for Aspect-Based Sentiment Analysis
    Zhao, Narisa
    Gao, Huan
    Wen, Xin
    Li, Hui
    IEEE ACCESS, 2021, 9 : 15561 - 15569
  • [50] A Convolutional Neural Network Based Sentiment Classification and the Convolutional Kernel Representation
    Gao, Shen
    Zhang, Huaping
    Gao, Kai
    NATURAL LANGUAGE PROCESSING AND INFORMATION SYSTEMS, NLDB 2017, 2017, 10260 : 287 - 291