Aspect-aware semantic feature enhanced networks for multimodal aspect-based sentiment analysis

Multimodal aspect-based sentiment analysis aims to predict the sentiment polarity of all aspect targets from text-image pairs. Most existing methods fail to extract fine-grained visual sentiment information, leading to alignment issues between the two modalities due to inconsistent granularity. In addition, the deep interaction between syntactic structure and semantic information is also ignored. In this paper, we propose an Aspect-aware Semantic Feature Enhancement Network (ASFEN) for multimodal aspect-based sentiment analysis to learn aspect-aware semantic and sentiment information in images and texts. Specifically, images are converted into textual information with fine-grained emotional cues. We construct dependency syntax trees and multi-layer syntax masks to fuse syntactic and semantic information through graph convolution. Extensive experiments on two multimodal Twitter datasets demonstrate the superiority of ASFEN over existing methods. The code is publicly available at https://github.com/lllppi/ASFEN. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2024.