Sentiment analysis method of consumer reviews based on multi-modal feature mining

Traditional sentiment analysis methods primarily rely on textual data. However, in real-world applications, product reviews often contain multimodal information including images, videos, and audio. This multimodal data is crucial for accurately understanding consumer sentiment trends. Therefore, this article proposes a novel product review sentiment analysis method that leverages multimodal feature mining. Firstly, a pre-trained vision transformer (VIT) and baidu general embedding (BGE) models are utilized to encode image and text features. Then, internal correlation mining is performed on image and text features through cross-attention. Next, the image and text features are aligned using a mask graph transformer network. The final encoding is obtained using multi-layer perceptron. Lastly, cosine similarity is computed between this encoding and each sentiment aspect of BGE encoding to determine corresponding sentiment scores. Simulations were conducted on a multi-modal Multi-ZOL dataset to compare the proposed method with several state-of-the-art techniques. The obtained results validated the superior performance of the proposed method. © 2024