Leveraging vision transformers and entropy-based attention for accurate micro-expression recognition

Micro-expressions are difficult to fake and inherently truthful, making micro-expression recognition technology widely applicable across various domains. With the development of artificial intelligence, the accuracy and efficiency of micro-expression recognition systems have been significantly improved. However, the short duration and subtle facial movement changes present significant challenges to real-time recognition and accuracy. To address these issues, this paper proposes a novel micro-expression recognition method based on the Vision Transformer. First, a new model called HTNet with LAPE (hierarchical transformer network with learnable absolute position embedding) is introduced to improve the model’s capacity for capturing subtle facial features, thereby enhancing the accuracy of micro-expression recognition. Second, an entropy-based selection agent attention is proposed to reduce the model parameters and computational effort while preserving its learning capability. Finally, a diffusion model is utilized for data augmentation to expand the micro-expression sample size, further enhancing the model’s generalization, accuracy, and robustness. Extensive experiments conducted on multiple datasets validate the framework’s effectiveness and highlight its potential in real-world applications.