FaST-Net: Face Style Self-Transfer Network for masked face inpainting

Yiming Li; Jianpeng Chen; Yazhou Ren; Xiaorong Pu

doi:10.1117/12.2673410

3 April 2023 FaST-Net: Face Style Self-Transfer Network for masked face inpainting

Yiming Li, Jianpeng Chen, Yazhou Ren, Xiaorong Pu

Proceedings Volume 12599, Second International Conference on Digital Society and Intelligent Systems (DSInS 2022); 1259925 (2023) https://doi.org/10.1117/12.2673410
Event: 2nd International Conference on Digital Society and Intelligent Systems (DSInS 2022), 2022, Chendgu, China

Abstract

During the COVID-19 coronavirus epidemic, people usually wear masks to prevent the spread of the virus, which has become a major obstacle when we use face-based computer vision techniques such as face recognition and face detection. So masked face inpainting technique is desired. Actually, the distribution of face features is strongly correlated with each other, but existing inpainting methods typically ignore the relationship between face feature distributions. To address this issue, in this paper, we first show that the face image inpainting task can be seen as a distribution alignment between face features in damaged and valid regions, and style transfer is a distribution alignment process. Based on this theory, we propose a novel face inpainting model considering the probability distribution between face features, namely Face Style Self-Transfer Network (FaST-Net). Through the proposed style self-transfer mechanism, FaST-Net can align the style distribution of features in the inpainting region with the style distribution of features in the valid region of a face. Ablation studies have validated the effectiveness of FaST-Net, and experimental results on two popular human face datasets (CelebA and VGGFace) exhibit its superior performance compared with existing state-of-the-art methods.

Citation Download Citation

Yiming Li, Jianpeng Chen, Yazhou Ren, and Xiaorong Pu "FaST-Net: Face Style Self-Transfer Network for masked face inpainting", Proc. SPIE 12599, Second International Conference on Digital Society and Intelligent Systems (DSInS 2022), 1259925 (3 April 2023); https://doi.org/10.1117/12.2673410

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image segmentation

Convolution

Ablation

Feature extraction

Image processing

Modulation

Visualization

Show All Keywords

Keywords/Phrases

Search In:

Publication Years