ReliableSwap: Boosting General Face Swapping Via Reliable Supervision

     

Ge Yuan^1,2,+, Maomao Li^2,+, Yong Zhang^2,*, Huicheng Zheng^1,* (+ Equal Contributions, * Corresponding Authors)

¹ Sun Yat-sen University     ² Tencent AI Lab    

TL;DR: A general face swapping framework that:

🎯 solves no image-level guidance
👩‍❤️‍👩 enhances source identity preservation
♾️ is orthogonal and compatible with existing methods

Updates

• 2023/06/25: Code released!

What Problems We Solve

During face swapping training, the re-construction task (used when $X_{\rm{t}}=X_{\rm{s}}$) cannot be used as the proxy anymore when $X_{\rm{t}} \neq X_{\rm{s}}$, lacking pixel-wise supervison $\Gamma$.

How It Works

We first use real images $C_{\rm{a}}$ and $C_{\rm{b}}$ to synthesize fake images $C_{\rm{ab}}$ and $C_{\rm{ba}}$. This synthesizing stage preserves the true source identity and target attributes based on Face Reenactment, Multi-Band Blending, and Face Reshaping.

Then based on the cycle relationship, for face swapping training stage, we use fake images as inputs while real images as pixel-level supervisons $\Gamma$, keeping the output domain close to the real and natural distribution and solving the non-supervision issue. In this way, the trainable face swapping network is guided to generate source identity-consistency swapping results, while also keeping target attributes.

More details can be found in our project page.

Usage

1. Environment Preparation
2. Training
3. Testing
4. Constructing Naive/Cycle Triplets by Yourself (Optional)

TODO

• release code
• extending to $512^2$ resolution

BibTex

@article{yuan2023reliableswap,
    title={ReliableSwap: Boosting General Face Swapping Via Reliable Supervision},
    author={Yuan, Ge and Li, Maomao and Zhang, Yong and Zheng, Huicheng},
    journal={arXiv preprint arXiv:2306.05356},
    year={2023}
}

📢 Disclaimer

This is not an official product of Tencent. This repository can only be used for personal/research/non-commercial purposes.

Free free to contact us if you feel uncomfortable.