Self-Attentive Contrastive Learning for Conditioned Periocular and Face Biometrics

Periocular and face are two common biometric modalities for identity management. Recently, the emergence of conditional biometrics has enabled the exploitation of the correlation between face and periocular to enhance each modality's performance, in which we coin intra-modal matching in this paper. However, limitations arise in each modality, particularly when wearing sunglasses or helmets, causing the absence of periocular or facial occlusion. A biometric system empowered with inter-modal matching capability between periocular and face is essential to mitigate these challenges. This paper presents a novel reciprocal learning model that utilizes periocular and face conditioning to facilitate flexible intra-modal and inter-modal matching. To address the intra-modal matching challenge, we devise a lightweight Gated Convolutional Channel-wise Self-Attention Network that enables selective attention to shared salient periocular and face features. On the other hand, to bridge the modality gap without sacrificing the intra-modal matching performance, we propose a modality and augmentation-aware contrastive loss that incorporates semi-supervised positive sampling and alignment-specific logit rescaling. Extensive identification and verification experiments on five face-periocular datasets under the open-set protocol attest to the efficacy of our proposed methods. Codes are publicly available at https://github.com/tiongsikng/gc2sa_net. © 2005-2012 IEEE.