Version 1
: Received: 8 February 2020 / Approved: 10 February 2020 / Online: 10 February 2020 (10:16:37 CET)
How to cite:
Yang, J.; Qi, Z.; Shi, Y. Learning to Incorporate Structure Knowledge for Image Inpainting. Preprints2020, 2020020125. https://doi.org/10.20944/preprints202002.0125.v1
Yang, J.; Qi, Z.; Shi, Y. Learning to Incorporate Structure Knowledge for Image Inpainting. Preprints 2020, 2020020125. https://doi.org/10.20944/preprints202002.0125.v1
Yang, J.; Qi, Z.; Shi, Y. Learning to Incorporate Structure Knowledge for Image Inpainting. Preprints2020, 2020020125. https://doi.org/10.20944/preprints202002.0125.v1
APA Style
Yang, J., Qi, Z., & Shi, Y. (2020). Learning to Incorporate Structure Knowledge for Image Inpainting. Preprints. https://doi.org/10.20944/preprints202002.0125.v1
Chicago/Turabian Style
Yang, J., Zhiquan Qi and Yong Shi. 2020 "Learning to Incorporate Structure Knowledge for Image Inpainting" Preprints. https://doi.org/10.20944/preprints202002.0125.v1
Abstract
This paper develops a multi-task learning framework that attempts to incorporate the image structure knowledge to assist image inpainting, which is not well explored in previous works. The primary idea is to train a shared generator to simultaneously complete the corrupted image and corresponding structures --- edge and gradient, thus implicitly encouraging the generator to exploit relevant structure knowledge while inpainting. In the meantime, we also introduce a structure embedding scheme to explicitly embed the learned structure features into the inpainting process, thus to provide possible preconditions for image completion. Specifically, a novel pyramid structure loss is proposed to supervise structure learning and embedding. Moreover, an attention mechanism is developed to further exploit the recurrent structures and patterns in the image to refine the generated structures and contents. Through multi-task learning, structure embedding besides with attention, our framework takes advantage of the structure knowledge and outperforms several state-of-the-art methods on benchmark datasets quantitatively and qualitatively.
image inpainting; image completion; attention; pyramid structure loss; deep learning
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
