DeblurSR: Event-Based Motion Deblurring under the Spiking Representation

被引:0
|
作者
Song, Chen [1 ]
Bajaj, Chandrajit [1 ]
Huang, Qixing [1 ]
机构
[1] Univ Texas Austin, Austin, TX 78712 USA
关键词
NETWORKS;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We present DeblurSR, a novel motion deblurring approach that converts a blurry image into a sharp video. DeblurSR utilizes event data to compensate for motion ambiguities and exploits the spiking representation to parameterize the sharp output video as a mapping from time to intensity. Our key contribution, the Spiking Representation (SR), is inspired by the neuromorphic principles determining how biological neurons communicate with each other in living organisms. We discuss why the spikes can represent sharp edges and how the spiking parameters are interpreted from the neuromorphic perspective. DeblurSR has higher output quality and requires fewer computing resources than state-of-the-art event-based motion deblurring methods. We additionally show that our approach easily extends to video super-resolution when combined with recent advances in implicit neural representation.
引用
收藏
页码:4900 / 4908
页数:9
相关论文
共 50 条
  • [1] Motion-Oriented Hybrid Spiking Neural Networks for Event-Based Motion Deblurring
    Liu, Zhaoxin
    Wu, Jinjian
    Shi, Guangming
    Yang, Wen
    Dong, Weisheng
    Zhao, Qinghang
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2024, 34 (05) : 3742 - 3754
  • [2] Learning Event-Based Motion Deblurring
    Jiang, Zhe
    Zhang, Yu
    Zou, Dongqing
    Ren, Jimmy
    Lv, Jiancheng
    Liu, Yebin
    [J]. 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, : 3317 - 3326
  • [3] Event-Based Shutter Unrolling and Motion Deblurring in Dynamic Scenes
    Wang, Yangguang
    Jiang, Chenxu
    Jia, Xu
    Guo, Yufei
    Yu, Lei
    [J]. IEEE SIGNAL PROCESSING LETTERS, 2024, 31 : 1069 - 1073
  • [4] Event-based Motion Deblurring with Modality-Aware Decomposition and Recomposition
    Yang, Wen
    Wu, Jinjian
    Li, Leida
    Dong, Weisheng
    Shi, Guangming
    [J]. PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 8327 - 8335
  • [5] Generalizing Event-Based Motion Deblurring in Real-World Scenarios
    Zhang, Xiang
    Yu, Lei
    Yang, Wen
    Liu, Jianzhuang
    Xia, Gui-Song
    [J]. 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2023, : 10700 - 10710
  • [6] Event-Based Fusion for Motion Deblurring with Cross-modal Attention
    Sun, Lei
    Sakaridis, Christos
    Liang, Jingyun
    Jiang, Qi
    Yang, Kailun
    Sun, Peng
    Ye, Yaozu
    Wang, Kaiwei
    Van Gool, Luc
    [J]. COMPUTER VISION - ECCV 2022, PT XVIII, 2022, 13678 : 412 - 428
  • [7] Event-Based Regression with Spiking Networks
    Guerrero, Elisa
    Quintana, Fernando M.
    Guerrero-Lebrero, Maria P.
    [J]. ADVANCES IN COMPUTATIONAL INTELLIGENCE, IWANN 2023, PT II, 2023, 14135 : 617 - 628
  • [8] Bio-inspired Event-based Motion Analysis with Spiking Neural Networks
    Oudjail, Veis
    Martinet, Jean
    [J]. VISAPP: PROCEEDINGS OF THE 14TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER VISION, IMAGING AND COMPUTER GRAPHICS THEORY AND APPLICATIONS, VOL 4, 2019, : 389 - 394
  • [9] Event-based Action Recognition Using Motion Information and Spiking Neural Networks
    Liu, Qianhui
    Xing, Dong
    Tang, Huajin
    Ma, De
    Pan, Gang
    [J]. PROCEEDINGS OF THE THIRTIETH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2021, 2021, : 1743 - 1749
  • [10] Event-based video deblurring based on image and event feature fusion
    Kim, Jeongmin
    Ghosh, Dipon Kumar
    Jung, Yong Ju
    [J]. EXPERT SYSTEMS WITH APPLICATIONS, 2023, 223