MMA-Net: Multi-view mixed attention mechanism for facial action unit detection

Facial action units (AU) have strong mutual correlation. How to explore fine-grained AU regional features from different dimensions while adding inter-AU correlational information is the key to accurate AU detection. In this paper, we propose a novel AU detection framework called MMA-Net based on multi view mixed attention, combining AU-regional information, co-occurrence correlational information and spatially correlational information by adopting a new AU partitioning scheme. Specifically, the proposed multi-view AU partitioning scheme first applies in both the AU co-occurrence correlational view and the facial ROI view to define the co-occurrence and spatially correlational information of AUs. Then, mixed attention, consisting of regional, channel-wise, and spatial attention, is incorporated into the encoder of MMA-Net to extract features from different dimensions. Finally, a pixel-level cross-view contrastive loss is proposed for feature enhancement by differing cross-view features for complement. Experimental results on two widely-used benchmark datasets, namely DISFA and BP4D, demonstrate the superior performance of MMA-Net against the state-of-the-art methods for AU detection.& COPY; 2023 Published by Elsevier B.V.