End-to-End Video Compressive Sensing Using Anderson-Accelerated Unrolled Networks

被引:14
|
作者
Li, Yuqi [1 ]
Qi, Miao [1 ]
Gulve, Rahul [2 ]
Wei, Mian [2 ]
Genov, Roman [2 ]
Kutulakos, Kiriakos N. [2 ]
Heidrich, Wolfgang [1 ]
机构
[1] KAUST, VCC Imaging Grp, Thuwal, Saudi Arabia
[2] Univ Toronto, Toronto, ON, Canada
基金
加拿大自然科学与工程研究理事会;
关键词
high-frame-rate imaging; deep neural network; computational camera; HIGH-SPEED; CODED EXPOSURE; RECONSTRUCTION; DESIGN;
D O I
10.1109/iccp48838.2020.9105237
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
Compressive imaging systems with spatial-temporal encoding can be used to capture and reconstruct fast-moving objects. The imaging quality highly depends on the choice of encoding masks and reconstruction methods. In this paper, we present a new network architecture to jointly design the encoding masks and the reconstruction method for compressive high-frame-rate imaging. Unlike previous works, the proposed method takes full advantage of denoising prior to provide a promising frame reconstruction. The network is also flexible enough to optimize full-resolution masks and efficient at reconstructing frames. To this end, we develop a new dense network architecture that embeds Anderson acceleration, known from numerical optimization, directly into the neural network architecture. Our experiments show the optimized masks and the dense accelerated network respectively achieve 1.5 dB and 1 dB improvements in PSNR without adding training parameters. The proposed method outperforms other state-of-the-art methods both in simulations and on real hardware. In addition, we set up a coded two-bucket camera for compressive high-frame-rate imaging, which is robust to imaging noise and provides promising results when recovering nearly 1,000 frames per second.
引用
收藏
页数:12
相关论文
共 50 条
  • [21] End-To-End Memory Networks
    Sukhbaatar, Sainbayar
    Szlam, Arthur
    Weston, Jason
    Fergus, Rob
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 28 (NIPS 2015), 2015, 28
  • [22] End-to-end variational quantum sensing
    Maclellan, Benjamin
    Roztocki, Piotr
    Czischek, Stefanie
    Melko, Roger G.
    NPJ QUANTUM INFORMATION, 2024, 10 (01)
  • [23] Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems
    Hung Le
    Sahoo, Doyen
    Chen, Nancy F.
    Hoi, Steven C. H.
    57TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2019), 2019, : 5612 - 5623
  • [24] END-TO-END CONTINUOUS EMOTION RECOGNITION FROM VIDEO USING 3D CONVLSTM NETWORKS
    Huang, Jian
    Li, Ya
    Tao, Jianhua
    Lian, Zheng
    Yi, Jiangyan
    2018 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2018, : 6837 - 6841
  • [25] An Automated End-To-End Pipeline for Fine-Grained Video Annotation using Deep Neural Networks
    Vandersmissen, Baptist
    Sterckx, Lucas
    Demeester, Thomas
    Jalalyand, Azarakhsh
    De Neye, Wesley
    Van de Walle, Rik
    ICMR'16: PROCEEDINGS OF THE 2016 ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL, 2016, : 409 - 412
  • [26] A new end-to-end QoS mechanism for video delivery over heterogeneous networks
    Shirwadkar, Ujwal
    Chilamkurti, Naveen
    Ke, Chih-Heng
    Shieh, C. K.
    2006 IFIP INTERNATIONAL CONFERENCE ON WIRELESS AND OPTICAL COMMUNICATIONS NETWORKS, 2006, : 143 - +
  • [27] A half-precision compressive sensing framework for end-to-end person re-identification
    Liao, Longlong
    Yang, Zhibang
    Liao, Qing
    Li, Kenli
    Li, Keqin
    Liu, Jie
    Tian, Qi
    NEURAL COMPUTING & APPLICATIONS, 2020, 32 (04): : 1141 - 1155
  • [28] A half-precision compressive sensing framework for end-to-end person re-identification
    Longlong Liao
    Zhibang Yang
    Qing Liao
    Kenli Li
    Keqin Li
    Jie Liu
    Qi Tian
    Neural Computing and Applications, 2020, 32 : 1141 - 1155
  • [29] End-to-End Neural Video Coding Using a Compound Spatiotemporal Representation
    Liu, Haojie
    Lu, Ming
    Chen, Zhiqi
    Cao, Xun
    Ma, Zhan
    Wang, Yao
    IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2022, 32 (08) : 5650 - 5662
  • [30] Online Compressive Transformer for End-to-End Speech Recognition
    Leong, Chi-Hang
    Huang, Yu-Han
    Chien, Jen-Tzung
    INTERSPEECH 2021, 2021, : 2082 - 2086