Enhancing credibility assessment in online social networks using multimodal deep learning

Nowadays, users post digital content indifferent modes over online platforms from anywhere and anytime. This unverified content can be consumed quickly by a wider audience and may influence them. Researchers have suggested various models to determine the credibility of user-generated content (UGC) on online platforms. Most of these approaches consider a single mode of data, i.e., either text or image, whereas UGC on most online platforms is multimodal. However, developing an efficient multimodal framework is still an open research challenge. This paper presents a Multimodal Credibility Assessment Framework (MmCAF) to classify the given user content into credible or not credible with minimal error. It concentrates on sentence-level embeddings to represent entire sentences and their semantic information as vectors. Furthermore, to extract complex image features, MmCAF uses transfer learning and compound model-scaling-based deep learning networks. Several experiments have been performed on three datasets to validate the efficacy of the proposed framework. In addition, extensive experiments with respect to fusion strategies and comparison with baseline models have been carried out to ascertain model effectiveness. The comparison of MmCAF with other state-of-the-art models shows that MmCAF has a significant performance gain of a minimum of 6% over other state-of-the-arts.