computer vision deep learning inpainting

Globally and Locally Consistent Image Completion

inpainting

Posted by stephen_zhou on October 7, 2017

Related work

  • diffusion-based image synthesis: histograms of local features
  • patch-based approaches: PatchMatch
  • CNN-based: limited to very small and thin masks
  • context encoder-based approach: based on GAN Figure 1: network of context encoder-based approach

Convolutional neural networks

  • A completion network

  • Two additional networks: the global and the local context discriminator networks

  • dilated convolution: 99x99 to 307x307 Figure 2: network of this paper

Loss function

  • MSE loss:
  • GAN loss:
  • Total loss:

Trianing algorithm

Experiment

Comparison with ExistingWork

Arbitrary Region Completion

Center Region Completion

Global and Local Consistency

Object Removal

Faces and Facades

dataset:

  • CelebFaces Attributes Dataset: 202, 599
  • the CMP Facade dataset: 606

Limitations and Discussion

  • square masks, especially when inpainting mask is at the border of the image

  • heavily structured object, is partially masked ** Figure 3:** User study on the [Hays and Efros 2007] dataset. We compare Ground Truth (GT) images, [Hays and Efros 2007] (Hays), CE [Pathak et al. 2016], and our approach

CATALOG
    FEATURED TAGS
    machine learning generative model computer vision deep learning GAN VAE object detection Video Understanding Generative Model RNN Neural Networks Neural Network Image Synthesis
    FRIENDS