Edge consistent image completion based on multi-granularity feature fusion

被引：0
|
作者：
Zhang S.-Y. [1 ]
Wang G.-Y. [1 ]
Liu Q. [1 ]
Wang R.-Q. [1 ]
机构：
[1] School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing
来源：
Kongzhi yu Juece/Control and Decision | 2022年 / 37卷 / 12期
关键词：
deep learning; edge discriminator; generative adversarial networks; image completion; multi-granularity cognitive computing;
D O I：
10.13195/j.kzyjc.2021.0665
中图分类号：
学科分类号：
摘要：
Image completion is an important research content in the field of digital image processing, and the completion of large area irregular missing images is a research hotspot in recent years. However, the existing image completion technology has some limitations. The method based on generative adversarial network ignores the edge structure information of the image, and it can’t restore the fine details. The method based on local discriminator can’t deal with the missing irregular image, and the completion task doesn’t conform to the actual application scene. Combined with the idea of multi-granularity cognitive computing, this paper proposes an edge discriminator based on multi-granularity feature fusion, which can fully learn the edge structure information of different granularity, improve the consistency between the generated image edge and the real image edge, and generate the complete image with clearer structure. At the same time, the edge space attenuation loss is introduced, which can improve the continuity of edge pixels. In addition, the attention mechanism is used to optimize the local discriminator to process the irregular missing image. Experimental results on Places 2, Paris Streetview and other public datasets show that the proposed method achieves better results than other image completion methods in the completion of large areas of irregular missing images, which illustrates the importance of edge structure information in image completion research to a certain extent. © 2022 Northeast University. All rights reserved.
引用
收藏
页码：3240 / 3250
页数：10
相关论文
共 37 条
  • [1] Bertalmio M, Sapiro G, Caselles V, Et al., Image inpainting, Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417-424, (2000)
  • [2] Criminisi A, Perez P, Toyama K., Object removal by exemplar-based inpainting[C], IEEE Computer Society Conference on Computer Vision and Pattern Recognitio Proceedings, (2003)
  • [3] Wang Y X., Novel approaches in cognitive informatics and natural intelligence, Scitech Book News, 33, 1, pp. 65-78, (2009)
  • [4] Lecun Y, Bottou L, Bengio Y, Et al., Gradient-based learning applied to document recognition, Proceedings of the IEEE, 86, 11, pp. 2278-2324, (1998)
  • [5] Vinyals O, Toshev A, Bengio S, Et al., Show and tell: A neural image caption generator, IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156-3164, (2015)
  • [6] Song Y H, Yang C, Lin Z, Et al., Contextual-based image inpainting: Infer, match, and translate, Proceedings of the European Conference on Computer Vision, pp. 3-18, (2018)
  • [7] Mirza M, Osindero S, Conditional generative adversarial nets, (2018)
  • [8] Yan Z, Li X, Li M, Et al., Shift-net: Image inpainting via deep feature rearrangement, Proceedings of the European Conference on Computer Vision, pp. 1-17, (2018)
  • [9] Liu G, Reda F A, Shih K J, Et al., Image inpainting for irregular holes using partial convolutions, Proceedings of the European Conference on Computer Vision (ECCV), pp. 89-105, (2018)
  • [10] Eitz M, Hays J, Alexa M., How do humans sketch objects?, ACM Transactions on Graphics, 31, 4, pp. 1-10, (2012)
1234