End-to-End Video Compressive Sensing Using Anderson-Accelerated Unrolled Networks

被引:14
|
作者
Li, Yuqi [1 ]
Qi, Miao [1 ]
Gulve, Rahul [2 ]
Wei, Mian [2 ]
Genov, Roman [2 ]
Kutulakos, Kiriakos N. [2 ]
Heidrich, Wolfgang [1 ]
机构
[1] KAUST, VCC Imaging Grp, Thuwal, Saudi Arabia
[2] Univ Toronto, Toronto, ON, Canada
基金
加拿大自然科学与工程研究理事会;
关键词
high-frame-rate imaging; deep neural network; computational camera; HIGH-SPEED; CODED EXPOSURE; RECONSTRUCTION; DESIGN;
D O I
10.1109/iccp48838.2020.9105237
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
Compressive imaging systems with spatial-temporal encoding can be used to capture and reconstruct fast-moving objects. The imaging quality highly depends on the choice of encoding masks and reconstruction methods. In this paper, we present a new network architecture to jointly design the encoding masks and the reconstruction method for compressive high-frame-rate imaging. Unlike previous works, the proposed method takes full advantage of denoising prior to provide a promising frame reconstruction. The network is also flexible enough to optimize full-resolution masks and efficient at reconstructing frames. To this end, we develop a new dense network architecture that embeds Anderson acceleration, known from numerical optimization, directly into the neural network architecture. Our experiments show the optimized masks and the dense accelerated network respectively achieve 1.5 dB and 1 dB improvements in PSNR without adding training parameters. The proposed method outperforms other state-of-the-art methods both in simulations and on real hardware. In addition, we set up a coded two-bucket camera for compressive high-frame-rate imaging, which is robust to imaging noise and provides promising results when recovering nearly 1,000 frames per second.
引用
收藏
页数:12
相关论文
共 50 条
  • [1] End-to-End Video Snapshot Compressive Imaging using Video Transformers
    Saideni, Wael
    Courreges, Fabien
    Helbert, David
    Cances, Jean Pierre
    2022 ELEVENTH INTERNATIONAL CONFERENCE ON IMAGE PROCESSING THEORY, TOOLS AND APPLICATIONS (IPTA), 2022,
  • [2] An END-To-END Image Compressive Sensing Algorithm Based on Attention Neural Networks
    Liu, Jiawei
    Gong, Jiulu
    Chen, Derong
    PROCEEDINGS OF 2020 3RD INTERNATIONAL CONFERENCE ON UNMANNED SYSTEMS (ICUS), 2020, : 745 - 750
  • [3] Training End-to-End Unrolled Iterative Neural Networks for SPECT Image Reconstruction
    Li, Zongyu
    Dewaraja, Yuni K.
    Fessler, Jeffrey A.
    IEEE TRANSACTIONS ON RADIATION AND PLASMA MEDICAL SCIENCES, 2023, 7 (04) : 410 - 420
  • [4] Image Restoration Based on End-to-End Unrolled Network
    Tao, Xiaoping
    Zhou, Hao
    Chen, Yueting
    PHOTONICS, 2021, 8 (09)
  • [5] End-to-End Deep Learning-Based Compressive Spectrum Sensing in Cognitive Radio Networks
    Meng, Xiangyue
    Inaltekin, Hazer
    Krongold, Brian
    ICC 2020 - 2020 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS (ICC), 2020,
  • [6] End-to-End Video-to-Speech Synthesis Using Generative Adversarial Networks
    Mira, Rodrigo
    Vougioukas, Konstantinos
    Ma, Pingchuan
    Petridis, Stavros
    Schuller, Bjoern W.
    Pantic, Maja
    IEEE TRANSACTIONS ON CYBERNETICS, 2023, 53 (06) : 3454 - 3466
  • [7] Image Compression Based on Compressive Sensing: End-to-End Comparison With JPEG
    Yuan, Xin
    Haimi-Cohen, Raziel
    IEEE TRANSACTIONS ON MULTIMEDIA, 2020, 22 (11) : 2889 - 2904
  • [8] MODELING VBR VIDEO OVER NETWORKS END-TO-END USING DETERMINISTIC SMOOTHING AT THE SOURCE
    SHROFF, N
    SCHWARTZ, M
    INTERNATIONAL JOURNAL OF COMMUNICATION SYSTEMS, 1994, 7 (04) : 337 - 348
  • [9] End-to-End Video Captioning
    Olivastri, Silvio
    Singh, Gurkirt
    Cuzzolin, Fabio
    2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW), 2019, : 1474 - 1482
  • [10] CSVideoNet: A Real-time End-to-end Learning Framework for High-frame-rate Video Compressive Sensing
    Xu, Kai
    Ren, Fengbo
    2018 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2018), 2018, : 1680 - 1688