A deep learning approach for image and text classification using neutrosophy

The proliferation of data on the web and on personal computers is a direct result of the proliferation of new technologies and gadgets. Most of these pieces of information are collected through a number of different methods (text, image, video, etc.). This type of information is also vital for e-commerce websites. The products on these websites feature both images and descriptions in text form, making them multimodal in nature. Earlier classification and information retrieval algorithms focused largely on a single modality. This study leverages multimodal data for categorization utilising neutrosophic fuzzy sets for uncertainty management for information retrieval tasks. This work employs image and text data and, inspired by prior ways of embedding text over an image, seeks to classify the images using neutrosophic classification algorithms. Neurosophic convolutional neural networks (NCNNs) are used to learn feature representations of the generated images for classification tasks. We demonstrate how a pipeline based on NCNN may be applied to learn representations of the unique fusion method. Traditional convolutional neural networks are subject to unexpected noisy conditions in the test phase, and as a result, their performance for the categorization of noisy data degrades. Comparing our technique against individual sources on multi-modal classification dataset provides good results. © The Author(s), under exclusive licence to Bharati Vidyapeeth's Institute of Computer Applications and Management 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.