Zhang, X.; Zhou, W.; Liu, K.; Tang, H.; Zhang, Z.; Zhang, W.; Yu, N. ControlFace: Feature Disentangling for Controllable Face Swapping. J. Imaging2024, 10, 21.
Zhang, X.; Zhou, W.; Liu, K.; Tang, H.; Zhang, Z.; Zhang, W.; Yu, N. ControlFace: Feature Disentangling for Controllable Face Swapping. J. Imaging 2024, 10, 21.
Zhang, X.; Zhou, W.; Liu, K.; Tang, H.; Zhang, Z.; Zhang, W.; Yu, N. ControlFace: Feature Disentangling for Controllable Face Swapping. J. Imaging2024, 10, 21.
Zhang, X.; Zhou, W.; Liu, K.; Tang, H.; Zhang, Z.; Zhang, W.; Yu, N. ControlFace: Feature Disentangling for Controllable Face Swapping. J. Imaging 2024, 10, 21.
Abstract
Face swapping is an intriguing and intricate task in the field of computer vision. Currently, most mainstream face swapping methods employ face recognition models to extract identity features and inject them into the generation process. Nonetheless, such methods often struggle to effectively transfer identity information, result in generated results failing to achieve a high identity similarity with the source face. Furthermore, if we can accurately disentangle identity information, we can achieve controllable face swapping, thereby providing more choices to users. In pursuit of this goal, we propose a new face swapping framework (ControlFace) based on the disentanglement of identity information. We disentangle the structure and texture of the source face, encoding and characterizing them in the form of feature embeddings separately. According to the semantic level of each feature representation, we inject them into the corresponding feature mapper and fuse them adequately in the latent space of StyleGAN. Owing to such disentanglement of structure and texture, we are able to controllablely transfer parts of the identity features. Extensive experiments and comparisons with state-of-the-art face swapping methods demonstrate the superiority of our face swapping framework in terms of transferring identity information, producing high-quality face images and controllable face swapping.
Keywords
face swapping; feature disentanglement; semantic hierarchy-based feature fusion; controllable identity feature transfer
Subject
Computer Science and Mathematics, Computer Vision and Graphics
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
