ARF-Net: a multi-modal aesthetic attention-based fusion

Over the last decade, Online Social Media platforms have witnessed a dramatic expansion due to the substantial reliance of individuals on these communication channels. These platforms are widely utilized to convey emotions, share opinions, and express preferences through various means such as artworks, multimedia contents, and blogs. Researchers are exploring these individual-specific traits for biometric identification. Aesthetic biometric systems utilize users' unique preferences across various subjective forms such as images, music, and textual contents. This study introduces a novel multi-modal aesthetic system, with a primary contribution to the development of an attention-based fusion method for person identification. The proposed identification system leverages a deep pre-trained model for high-level feature extraction from visual and auditory modalities. The paper introduces a novel fusion architecture named attention-based residual fusion network (ARF-Net) to incorporate two heterogeneous aesthetic feature vectors. The proposed model yielded a 99.38% identification accuracy on the Aesthetic Image Audio 32 (AIA32) dataset and 98.02% identification accuracy on Aesthetic Image Audio 52 (AIA52) dataset, outperforming other aesthetic biometric systems. The proposed architecture stands out for its efficiency, showcasing a lightweight architecture with minimal parameters, ensuring optimal performance in different modalities.