Deep Blind Hyperspectral Image Super-Resolution

被引:80
|
作者
Zhang, Lei [1 ]
Nie, Jiangtao [1 ]
Wei, Wei [1 ,2 ,3 ]
Li, Yong [1 ]
Zhang, Yanning [1 ,2 ]
机构
[1] Northwestern Polytech Univ, Sch Comp Sci, Shaanxi Prov Key Lab Speech & Image Informat Proc, Xian 710072, Peoples R China
[2] Northwestern Polytech Univ, Sch Comp Sci, Natl Engn Lab Integrated Aerosp Ground Ocean Big, Xian 710072, Peoples R China
[3] Northwestern Polytech Univ Shenzhen, Res & Dev Inst, Shenzhen 518057, Peoples R China
基金
中国国家自然科学基金;
关键词
Deep unsupervised learning; fusion-based hyperspectral image (HSI) super-resolution; unknown degeneration;
D O I
10.1109/TNNLS.2020.3005234
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The production of a high spatial resolution (HR) hyperspectral image (HSI) through the fusion of a low spatial resolution (LR) HSI with an HR multispectral image (MSI) has underpinned much of the recent progress in HSI super-resolution. The premise of these signs of progress is that both the degeneration from the HR HSI to LR HSI in the spatial domain and the degeneration from the HR HSI to HR MSI in the spectral domain are assumed to be known in advance. However, such a premise is difficult to achieve in practice. To address this problem, we propose to incorporate degeneration estimation into HSI super-resolution and present an unsupervised deep framework for "blind" HSIs super-resolution where the degenerations in both domains are unknown. In this framework, we model the latent HR HSI and the unknown degenerations with deep network structures to regularize them instead of using handcrafted (or shallow) priors. Specifically, we generate the latent HR HSI with an image-specific generator network and structure the degenerations in spatial and spectral domains through a convolution layer and a fully connected layer, respectively. By doing this, the proposed framework can be formulated as an end-to-end deep network learning problem, which is purely supervised by those two input images (i.e., LR HSI and HR MSI) and can be effectively solved by the backpropagation algorithm. Experiments on both natural scene and remote sensing HSI data sets show the superior performance of the proposed method in coping with unknown degeneration either in the spatial domain, spectral domain, or even both of them.
引用
收藏
页码:2388 / 2400
页数:13
相关论文
共 50 条
  • [21] Hyperspectral Image Super-Resolution Meets Deep Learning: A Survey and Perspective
    Wang, Xinya
    Hu, Qian
    Cheng, Yingsong
    Ma, Jiayi
    [J]. IEEE-CAA JOURNAL OF AUTOMATICA SINICA, 2023, 10 (08) : 1668 - 1691
  • [22] Hyperspectral Image Super-Resolution Meets Deep Learning: A Survey and Perspective
    Xinya Wang
    Qian Hu
    Yingsong Cheng
    Jiayi Ma
    [J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10 (08) : 1668 - 1691
  • [23] Hyperspectral Image Super-Resolution by Deep Spatial-Spectral Exploitation
    Hu, Jing
    Zhao, Minghua
    Li, Yunsong
    [J]. REMOTE SENSING, 2019, 11 (10)
  • [24] Hyperspectral Image Super-Resolution Using Deep Feature Matrix Factorization
    Xie, Weiying
    Jia, Xiuping
    Li, Yunsong
    Lei, Jie
    [J]. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2019, 57 (08): : 6055 - 6067
  • [25] Hyperspectral image super-resolution using deep convolutional neural network
    Li, Yunsong
    Hu, Jing
    Zhao, Xi
    Xie, Weiying
    Li, JiaoJiao
    [J]. NEUROCOMPUTING, 2017, 266 : 29 - 41
  • [26] Deep Blind Video Super-resolution
    Pan, Jinshan
    Bai, Haoran
    Dong, Jiangxin
    Zhang, Jiawei
    Tang, Jinhui
    [J]. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 4791 - 4800
  • [27] Deep Blind Video Super-resolution
    Pan, Jinshan
    Bai, Haoran
    Dong, Jiangxin
    Zhang, Jiawei
    Tang, Jinhui
    [J]. Proceedings of the IEEE International Conference on Computer Vision, 2021, : 4791 - 4800
  • [28] Efficient Blind Image Super-Resolution
    Vais, Olga
    Makarov, Ilya
    [J]. ADVANCES IN COMPUTATIONAL INTELLIGENCE, IWANN 2023, PT II, 2023, 14135 : 229 - 240
  • [29] Hyperspectral Image Super-Resolution With a Mosaic RGB Image
    Fu, Ying
    Zheng, Yinqiang
    Huang, Hua
    Sato, Imari
    Sato, Yoichi
    [J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, 2018, 27 (11) : 5539 - 5552
  • [30] ROBUST DEEP HYPERSPECTRAL IMAGERY SUPER-RESOLUTION
    Nie, Jiangtao
    Zhang, Lei
    Wang, Cong
    Wei, Wei
    Zhang, Yanning
    [J]. 2019 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM (IGARSS 2019), 2019, : 847 - 850