Beyond Text: Multimodal Credibility Assessment Approaches for Online User-Generated Content

User-generated content (UGC) is increasingly becoming prevalent on various digital platforms. The content generated on social media, review forums, and question-answer platforms impacts a larger audience and influences their political, social, and other cognitive abilities. Traditional credibility assessment mechanisms involve assessing the credibility of the source and the text. However, with the increase in how user content can be generated and shared (audio, video, and images), multimodal representation of UGC has become increasingly popular. This article reviews the credibility assessment of UGC in various domains, particularly identifying fake news, suspicious profiles, and fake reviews and testimonials, focusing on both textual content and the source of the content creator. Next, the concept of multimodal credibility assessment is presented, which also includes audio, video, and images in addition to text. After that, the article presents a systematic review and comprehensive analysis of work done in the credibility assessment of UGC considering multimodal features. Additionally, the article provides extensive details on the publicly available multimodal datasets for the credibility assessment of UGC. In the end, the research gaps, challenges, and future directions in assessing the credibility of multimodal UGC are presented. © 2024 Copyright held by the owner/author(s).