A Lightweight Image-Text Sentiment Analysis Model Based on Public Emotion Feature Compression and Fusion

被引：0
|
作者：
Gan, Chenquan [1 ]
Fu, Xiang [1 ]
Feng, Qingdong [1 ]
Zhu, Qingyi [2 ]
机构：
[1] School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing,400065, China
[2] School of Cyber Security and Information Law, Chongqing University of Posts and Telecommunications, Chongqing,400065, China
来源：
Jisuanji Yanjiu yu Fazhan/Computer Research and Development | 2023年 / 60卷 / 05期
关键词：
Due to the combination of image and text can better reflect the users’ attitude and standpoint; image-text sentiment analysis has become a research hotspot. However; the existing sentiment analysis methods cannot extract and fuse image-text emotion information effectively; which results in low performance; large amount of parameters; and difficulty in deployment. In this paper; a lightweight image-text sentiment analysis model using public emotion feature compression and fusion is proposed. This model designs the image and text feature compression module by combining the convolution layer and fully connected layer to extract and compress the feature for reducing the feature dimension simultaneously. In addition; a public emotion feature fusion module based on the gating mechanism is proposed to eliminate the heterogeneity of image-text features through mapping the image and text features to the same emotional space and reduce the redundant information by extracting and fusing the public emotion features of image-text. Experimental results on 3 baseline datasets of Twitter; Flickr; and Getty Images show that the proposed model can extract and fuse the emotional information of image-text more effectively than the early models. Compared with the latest models; the proposed model greatly reduces model parameters and has better performance; and is easier to be deployed. © 2023 Science Press. All rights reserved;
D O I：
10.7544/issn1000-1239.202111218
中图分类号：
学科分类号：
摘要：
引用
收藏
页码：1099 / 1110
相关论文
共 50 条
  • [1] Image-Text Fusion Sentiment Analysis Method Based on Image Semantic Translation
    Huang, Jian
    Wang, Ying
    [J]. Computer Engineering and Applications, 2023, 59 (11) : 180 - 187
  • [2] Multi-granularity Feature Attention Fusion Network for Image-Text Sentiment Analysis
    Sun, Tao
    Wang, Shuang
    Zhong, Shenjie
    [J]. ADVANCES IN COMPUTER GRAPHICS, CGI 2022, 2022, 13443 : 3 - 14
  • [3] Image-text interaction graph neural network for image-text sentiment analysis
    Wenxiong Liao
    Bi Zeng
    Jianqi Liu
    Pengfei Wei
    Jiongkun Fang
    [J]. Applied Intelligence, 2022, 52 : 11184 - 11198
  • [4] Image-text sentiment analysis via deep multimodal attentive fusion
    Huang, Feiran
    Zhang, Xiaoming
    Zhao, Zhonghua
    Xu, Jie
    Li, Zhoujun
    [J]. KNOWLEDGE-BASED SYSTEMS, 2019, 167 : 26 - 37
  • [5] Image-text interaction graph neural network for image-text sentiment analysis
    Liao, Wenxiong
    Zeng, Bi
    Liu, Jianqi
    Wei, Pengfei
    Fang, Jiongkun
    [J]. APPLIED INTELLIGENCE, 2022, 52 (10) : 11184 - 11198
  • [6] The Research on Image-text Fusion Emotion Recognition Based on Deep Learning
    Wang, Xuyang
    Zhang, Xin
    [J]. PROCEEDINGS OF 2024 INTERNATIONAL CONFERENCE ON COMPUTER AND MULTIMEDIA TECHNOLOGY, ICCMT 2024, 2024, : 124 - 128
  • [7] BIT: Improving Image-text Sentiment Analysis via Learning Bidirectional Image-text Interaction
    Xiao, Xingwang
    Pu, Yuanyuan
    Zhao, Zhengpeng
    Gu, Jinjing
    Xu, Dan
    [J]. 2023 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, IJCNN, 2023,
  • [8] Multimodal Sentiment Analysis With Image-Text Interaction Network
    Zhu, Tong
    Li, Leida
    Yang, Jufeng
    Zhao, Sicheng
    Liu, Hantao
    Qian, Jiansheng
    [J]. IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 : 3375 - 3385
  • [9] Multimodal Sentiment Analysis With Image-Text Correlation Modal
    Li, Yuxin
    Jiang, Shan
    Chaomurilige
    [J]. 2023 IEEE INTERNATIONAL CONFERENCES ON INTERNET OF THINGS, ITHINGS IEEE GREEN COMPUTING AND COMMUNICATIONS, GREENCOM IEEE CYBER, PHYSICAL AND SOCIAL COMPUTING, CPSCOM IEEE SMART DATA, SMARTDATA AND IEEE CONGRESS ON CYBERMATICS,CYBERMATICS, 2024, : 281 - 286
  • [10] A Vegetable Leaf Disease Identification Model Based on Image-Text Cross-Modal Feature Fusion
    Feng, Xuguang
    Zhao, Chunjiang
    Wang, Chunshan
    Wu, Huarui
    Miao, Yisheng
    Zhang, Jingjian
    [J]. FRONTIERS IN PLANT SCIENCE, 2022, 13
12345