Phase driven transformer for micro-expression recognition

Because of the brevity, unconsciousness and subtlety of micro-expression (ME), the scale of ME dataset is not large and the ME recognition (MER) rate is not high. In addition, most methods focus on the extraction of spatial features, but ignore the information of other domains. To solve the above problems, this paper proposes phase driven Transformer (PDT). The PDT generated amplitude and phase information from two networks and fused them for network training. By incorporating image features in the frequency domain, the richness and diversity of features are better increased, enabling the model to extract more effective information and solve the problem of unclear micro-expression features. To address the problem of small sample size, dense relative localization loss is adopted in this paper. The experiments are conducted on three public datasets: SMIC, SAMM, and CASME II. The results demonstrate that the PDT outperforms other methods.