Repository files navigation

This project attempted to achieve the paperA novel GAN-based network for unmasking ofmasked face. The modelis designed to remove the face-mask from facial image and inpaint the left-behind region basedon a novel GAN-network approach.

• Google Cloud Platform
• GPU (Nvidia Tesla T4)
• Python 3.8

Rather than using the traditional pix2pix U-Net method, in this work the model consists of two main modules,map module andediting module.In the first module, we detect the face-mask object and generate abinary segmentation map for data augmentation. In the second module, we train the modified U-Netwith two discriminators using masked image and binary segmentation map.

• For collecting the ground truth, we useFlickr-Faces-HQ Dataset (FFHQ).
• For creating the masked images, we useMaskTheFace to masking the ground truth.

In this work, I used around 4k paired images for training map module model, and around 20k images for training editing module model.

It is recommended to make anew virtual environment withPython 3.8 and install the dependencies. Following stepscan be taken to download and run the Face-mask inpainting streamlit webapp on local host

git clone https://github.com/daviddirethucus/Face-Mask_Inpainting.git

Since it is not permissable to push the model which is larger than 100MB on Github, so we provide a link to download our trained Facemask Inpainting models:Here

The path of the trained models should be located at:

/Face-Mask_Inpainting/models

The provided requirements.txt file consists the essential packages to install. Use the following command

cd Face-Mask_Inpaintingpip install -r requirements.txt

cd Face-Mask_Inpaintingstreamlit run main.py

Copy theLocal URL /Network URL and view it in your browser.

| | |

• A novel GAN-based network for unmasking of masked face
• Generation of Realistic Facemasked Faces With GANs
• U-Net: Convolutional Networks for Biomedical Image Segmentation
• Squeeze-and-Excitation Networks
• Rethinking Atrous Convolution for Semantic Image Segmentation
• Language Modeling with Gated Convolutional Networks
• Image Quality Assessment: From Error Visibility to Structural Similarity
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution

• https://github.com/moskomule/senet.pytorch/blob/master/senet/se_module.py#L4
• https://github.com/VainF/pytorch-msssim
• https://gist.github.com/alper111/8233cdb0414b4cb5853f2f730ab95a49