Hierarchical Vector-Quantized Variational Autoencoder and Vector Credibility Mechanism for High-Quality Image Inpainting

被引:0
|
作者
Li, Cheng [1 ]
Xu, Dan [1 ]
Chen, Kuai [2 ]
机构
[1] Yunnan Univ, Sch Informat Sci & Engn, Kunming 650106, Peoples R China
[2] Yunnan Univ, Sch Govt, Kunming 650106, Peoples R China
基金
中国国家自然科学基金;
关键词
image inpainting; VQ-VAE; vector credibility; codebook;
D O I
10.3390/electronics13101852
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Image inpainting infers the missing areas of a corrupted image according to the information of the undamaged part. Many existing image inpainting methods can generate plausible inpainted results from damaged images with the fast-developed deep-learning technology. However, they still suffer from over-smoothed textures or textural distortion in the cases of complex textural details or large damaged areas. To restore textures at a fine-grained level, we propose an image inpainting method based on a hierarchical VQ-VAE with a vector credibility mechanism. It first trains the hierarchical VQ-VAE with ground truth images to update two codebooks and to obtain two corresponding vector collections containing information on ground truth images. The two vector collections are fed to a decoder to generate the corresponding high-fidelity outputs. An encoder then is trained with the corresponding damaged image. It generates vector collections approximating the ground truth by the help of the prior knowledge provided by the codebooks. After that, the two vector collections pass through the decoder from the hierarchical VQ-VAE to produce the inpainted results. In addition, we apply a vector credibility mechanism to promote vector collections from damaged images and approximate vector collections from ground truth images. To further improve the inpainting result, we apply a refinement network, which uses residual blocks with different dilation rates to acquire both global information and local textural details. Extensive experiments conducted on several datasets demonstrate that our method outperforms the state-of-the-art ones.
引用
收藏
页数:17
相关论文
共 50 条
  • [21] Connectionist temporal classification loss for vector quantized variational autoencoder in zero-shot voice conversion
    Kang, Xiao
    Huang, Hao
    Hu, Ying
    Huang, Zhihua
    DIGITAL SIGNAL PROCESSING, 2021, 116
  • [22] Enhancing Hierarchical Vector Quantized Autoencoders for Image Synthesis Through Multiple Decoders
    Serez, Dario
    Cristani, Marco
    Murino, Vittorio
    Del Bue, Alessio
    Morerio, Pietro
    IMAGE ANALYSIS AND PROCESSING, ICIAP 2023, PT II, 2023, 14234 : 393 - 405
  • [23] Anomaly Detection in Optical Coherence Tomography Angiography (OCTA) with a Vector-Quantized Variational Auto-Encoder (VQ-VAE)
    Jebril, Hana
    Esengoenuel, Meltem
    Bogunovic, Hrvoje
    BIOENGINEERING-BASEL, 2024, 11 (07):
  • [24] Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes
    Bond-Taylor, Sam
    Hessey, Peter
    Sasaki, Hiroshi
    Breckon, Toby P.
    Willcocks, Chris G.
    COMPUTER VISION, ECCV 2022, PT XXIII, 2022, 13683 : 170 - 188
  • [25] Vector Quantized Variational Autoencoder-Based Compressive Sampling Method for Time Series in Structural Health Monitoring
    Liang, Ge
    Ji, Zhenglin
    Zhong, Qunhong
    Huang, Yong
    Han, Kun
    SUSTAINABILITY, 2023, 15 (20)
  • [26] Low-Dose CT Image Reconstruction using Vector Quantized Convolutional Autoencoder with Perceptual Loss
    Ramanathan, Shalini
    Ramasundaram, Mohan
    SADHANA-ACADEMY PROCEEDINGS IN ENGINEERING SCIENCES, 2023, 48 (02):
  • [27] Low-Dose CT Image Reconstruction using Vector Quantized Convolutional Autoencoder with Perceptual Loss
    Shalini Ramanathan
    Mohan Ramasundaram
    Sādhanā, 48
  • [28] A Hierarchical Approach for High-Quality and Fast Image Completion
    Thanh Trung Dang
    Beghdadi, Azeddine
    Larabi, Mohamed-Chaker
    KNOWLEDGE AND SYSTEMS ENGINEERING (KSE 2013), VOL 1, 2014, 244 : 11 - 21
  • [29] Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting
    Zeng, Yanhong
    Fu, Jianlong
    Chao, Hongyang
    Guo, Baining
    2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019), 2019, : 1486 - 1494
  • [30] What constitutes high-quality evidence for malaria vector control?
    Hawley, William A.
    Laihad, Ferdinand J.
    Ochomo, Eric
    Achee, Nicole L.
    Grieco, John P.
    Armistead, Jennifer S.
    Nahlen, Bernard L.
    LANCET, 2023, 402 (10404): : 773 - 774