A lexicon-based approach for sentiment analysis of multimodal content in tweets

Sentiment analysis (SA) is widely used in various applications such as online opinion gathering for policy directives in government, monitoring of customers and staff satisfaction in corporate bodies in politics and security structures for public tension monitoring. Recently, the field met new challenges where new algorithms must contend with highly unstructured sources for sentiment expressions emanating from online social media fora. This study proposes a lexicon-based procedure to implement tweets sentiment analysis with improved algorithms. To deal with sources devoid of syntactic and grammatical structure, the approach incorporates lexicon-based text pre-processing using lexicon features such as tokenisation, parts-of-speech (POS), stop word removal, stemming, normalisation, word frequency and count for context analysis. However, since data in social network sites that primarily express sentiments are in multimodal forms, an analysis of multimedia content is required (i.e. retrieving text from audio, image and video). Hence a novel lexicon-based methodology and framework for multimodal sentiment analysis of text congregated from audio, images, and videos have been proposed. The lexicon-based approach classified the contents as positive, negative or neutral. The experiment results from STS-Gold datasets (recall = 93.2%, precision = 81.9%, F1-score = 87.2%, Accuracy = 92.29%) show the efficacy of the proposed approach as compared with those of other state-of-the-art research studies.