Hierarchical Vector-Quantized Variational Autoencoder and Vector Credibility Mechanism for High-Quality Image Inpainting

被引:0
|
作者
Li, Cheng [1 ]
Xu, Dan [1 ]
Chen, Kuai [2 ]
机构
[1] Yunnan Univ, Sch Informat Sci & Engn, Kunming 650106, Peoples R China
[2] Yunnan Univ, Sch Govt, Kunming 650106, Peoples R China
基金
中国国家自然科学基金;
关键词
image inpainting; VQ-VAE; vector credibility; codebook;
D O I
10.3390/electronics13101852
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Image inpainting infers the missing areas of a corrupted image according to the information of the undamaged part. Many existing image inpainting methods can generate plausible inpainted results from damaged images with the fast-developed deep-learning technology. However, they still suffer from over-smoothed textures or textural distortion in the cases of complex textural details or large damaged areas. To restore textures at a fine-grained level, we propose an image inpainting method based on a hierarchical VQ-VAE with a vector credibility mechanism. It first trains the hierarchical VQ-VAE with ground truth images to update two codebooks and to obtain two corresponding vector collections containing information on ground truth images. The two vector collections are fed to a decoder to generate the corresponding high-fidelity outputs. An encoder then is trained with the corresponding damaged image. It generates vector collections approximating the ground truth by the help of the prior knowledge provided by the codebooks. After that, the two vector collections pass through the decoder from the hierarchical VQ-VAE to produce the inpainted results. In addition, we apply a vector credibility mechanism to promote vector collections from damaged images and approximate vector collections from ground truth images. To further improve the inpainting result, we apply a refinement network, which uses residual blocks with different dilation rates to acquire both global information and local textural details. Extensive experiments conducted on several datasets demonstrate that our method outperforms the state-of-the-art ones.
引用
收藏
页数:17
相关论文
共 50 条
  • [41] Vector Network Analyser-based Terahertz Solution for High-quality Medical Imaging
    Ahmed, Hanya T.
    Jones, Robert C.
    Dubrovka, Rostyslav
    Donnan, Robert
    Alomainy, Akram
    2022 47TH INTERNATIONAL CONFERENCE ON INFRARED, MILLIMETER AND TERAHERTZ WAVES (IRMMW-THZ 2022), 2022,
  • [42] High-quality initial codebook design method of vector quantisation using grouping strategy
    Ma, Xiaoxiao
    Pan, Zhibin
    Li, Yang
    Fang, Jie
    IET IMAGE PROCESSING, 2015, 9 (11) : 986 - 992
  • [43] DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning
    Wang, Yizhi
    Lian, Zhouhui
    ACM TRANSACTIONS ON GRAPHICS, 2021, 40 (06):
  • [44] ReVQ-VAE: A Vector Quantization-Variational Autoencoder for COVID-19 Chest X-Ray Image Recovery
    Tarhouni, Nesrine
    Fourati, Rahma
    Charfeddine, Maha
    Ben Amar, Chokri
    COMPUTATIONAL COLLECTIVE INTELLIGENCE, ICCCI 2023, 2023, 14162 : 753 - 765
  • [45] Recovering real-world scene: high-quality image inpainting using multi-exposed references
    Zhu, Z. J.
    Li, Z. G.
    Rahardja, S.
    Fraenti, P.
    ELECTRONICS LETTERS, 2009, 45 (25) : 1310 - 1311
  • [46] High-Quality Codebook Generation of Vector Quantization Using the HT-ABC-LBG Algorithm
    Huang, Shu-Chien
    JOURNAL OF INFORMATION SCIENCE AND ENGINEERING, 2018, 34 (01) : 81 - 102
  • [47] High-quality and interactive animations of 3D time-varying vector fields
    Helgeland, Anders
    Elboth, Thomas
    IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 2006, 12 (06) : 1535 - 1546
  • [48] Lentiviral vector induces high-quality memory T cells via dendritic cells transduction
    Ku, Min Wen
    Authie, Pierre
    Nevo, Fabien
    Souque, Philippe
    Bourgine, Maryline
    Romano, Marta
    Charneau, Pierre
    Majlessi, Laleh
    COMMUNICATIONS BIOLOGY, 2021, 4 (01)
  • [49] Lentiviral vector induces high-quality memory T cells via dendritic cells transduction
    Min Wen Ku
    Pierre Authié
    Fabien Nevo
    Philippe Souque
    Maryline Bourgine
    Marta Romano
    Pierre Charneau
    Laleh Majlessi
    Communications Biology, 4
  • [50] VecFontSDF: Learning to Reconstruct and Synthesize High-quality Vector Fonts via Signed Distance Functions
    Xia, Zeqing
    Xiong, Bojun
    Lian, Zhouhui
    2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR, 2023, : 1848 - 1857