Hierarchical Vector-Quantized Variational Autoencoder and Vector Credibility Mechanism for High-Quality Image Inpainting

被引:0
|
作者
Li, Cheng [1 ]
Xu, Dan [1 ]
Chen, Kuai [2 ]
机构
[1] Yunnan Univ, Sch Informat Sci & Engn, Kunming 650106, Peoples R China
[2] Yunnan Univ, Sch Govt, Kunming 650106, Peoples R China
基金
中国国家自然科学基金;
关键词
image inpainting; VQ-VAE; vector credibility; codebook;
D O I
10.3390/electronics13101852
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Image inpainting infers the missing areas of a corrupted image according to the information of the undamaged part. Many existing image inpainting methods can generate plausible inpainted results from damaged images with the fast-developed deep-learning technology. However, they still suffer from over-smoothed textures or textural distortion in the cases of complex textural details or large damaged areas. To restore textures at a fine-grained level, we propose an image inpainting method based on a hierarchical VQ-VAE with a vector credibility mechanism. It first trains the hierarchical VQ-VAE with ground truth images to update two codebooks and to obtain two corresponding vector collections containing information on ground truth images. The two vector collections are fed to a decoder to generate the corresponding high-fidelity outputs. An encoder then is trained with the corresponding damaged image. It generates vector collections approximating the ground truth by the help of the prior knowledge provided by the codebooks. After that, the two vector collections pass through the decoder from the hierarchical VQ-VAE to produce the inpainted results. In addition, we apply a vector credibility mechanism to promote vector collections from damaged images and approximate vector collections from ground truth images. To further improve the inpainting result, we apply a refinement network, which uses residual blocks with different dilation rates to acquire both global information and local textural details. Extensive experiments conducted on several datasets demonstrate that our method outperforms the state-of-the-art ones.
引用
收藏
页数:17
相关论文
共 50 条
  • [31] Two-Stage Generator Network for High-Quality Image Inpainting in Future Internet
    Zhao, Peng
    Zhang, Dan
    Geng, Shengling
    Zhou, Mingquan
    ELECTRONICS, 2023, 12 (06)
  • [32] A Vector Quantized Variational Autoencoder (VQ-VAE) Autoregressive Neural F0 Model for Statistical Parametric Speech Synthesis
    Wang, Xin
    Takaki, Shinji
    Yamagishi, Junichi
    King, Simon
    Tokuda, Keiichi
    IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2020, 28 (157-170) : 157 - 170
  • [33] High-quality animation of 2D steady vector fields
    Lefer, W
    Jobard, B
    Leduc, C
    IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 2004, 10 (01) : 2 - 14
  • [34] High-quality vector vortex arrays by holographic and geometric phase control
    Tang, Yuei
    Perrie, Walter
    Schille, Joerg
    Loeschner, Udo
    Li, Qianliang
    Liu, Dun
    Edwardson, Stuart P.
    Forbes, Andrew
    Dearden, Geoff
    JOURNAL OF PHYSICS D-APPLIED PHYSICS, 2020, 53 (46)
  • [35] 466.4-Gbit/s PCS-64QAM wireless transmission enhanced by dual-polarized SISO link and vector-quantized variational autoencoders
    Liu, Xiang
    Zhang, Jiao
    Zhu, Min
    Xin, Zhigang
    Tong, Wei Dong
    Wang, Yunwu
    Cai, Yuancheng
    Hua, Bingchang
    Lei, Mingzheng
    Liu, Bo
    Yu, Jianjun
    OPTICS EXPRESS, 2024, 32 (17): : 29379 - 29393
  • [36] HIGH-QUALITY COMPRESSION OF ECHOGRAPHIC IMAGES BY NEURAL NETWORKS AND VECTOR QUANTIZATION
    CONFORTO, S
    DALESSIO, T
    ZANETTE, E
    MANCINI, G
    MEDICAL & BIOLOGICAL ENGINEERING & COMPUTING, 1995, 33 (05) : 695 - 698
  • [37] Adaptive-hierarchical-filtering technique for high-quality magazine image reproduction
    Shu, J
    Lin, TN
    Bhattacharjya, A
    Shiozaki, T
    HUMAN VISION AND ELECTRONIC IMAGING V, 2000, 3959 : 625 - 633
  • [38] Adaptive-Hierarchical-Filtering Technique for High-Quality Magazine Image Reproduction
    Tsung Nan Lin
    Joseph Shu
    Journal of VLSI signal processing systems for signal, image and video technology, 2005, 39 : 237 - 247
  • [39] Adaptive-hierarchical-filtering technique for high-quality magazine image reproduction
    Lin, TN
    Shu, J
    JOURNAL OF VLSI SIGNAL PROCESSING SYSTEMS FOR SIGNAL IMAGE AND VIDEO TECHNOLOGY, 2005, 39 (03): : 237 - 247
  • [40] Laser-Material Interactions of High-Quality Ultrashort Pulsed Vector Vortex Beams
    Tang, Yue
    Perrie, Walter
    Sierra, David Rico
    Li, Qianliang
    Liu, Dun
    Edwardson, Stuart P.
    Dearden, Geoff
    MICROMACHINES, 2021, 12 (04)