A high-fidelity face swapping algorithm based on mutual information-guided feature decoupling

A large number of high-quality face swapping images are often required to improve the performance of forgery detection. High-quality face swapping images require fewer face swapping traces and fewer face swapping artificialities in the image while ensuring the same identity consistency with the source face and the same attribute consistency with the target. It is a challenging task today to properly separate identity and non-identity related attribute information. In this paper, we propose a novel framework that called high-fidelity face swapping algorithm (HFSA) which consists of two part networks, a GAN-based mutual information swapping network, MuIn-swap, for face swapping and an MAE-based Detail Repair Net, DRN, for detail repair. We introduce mutual information into feature compression, explicitly computing the mutual information of identity and attribute information obtained by compression of latent features. Therefore, the learning of the network is explicitly guided by formulating the minimum mutual information as the optimization goal, so that we can obtain pure identity and attribute information. In addition to overcome the problem of information loss during face swapping, we additionally design an DRN to repair the details of the face swapping to achieve more realistic and fidelity face swapping images. Through extensive experiments, it has been demonstrated that the forgery samples generated by HFSA guarantee smaller forgery traces and fewer artifacts while guaranteeing identity consistency with the source and attribute preservation. The code is already available on GitHub: https://github.com/karasuma123/HFSA.