Blurry Video Frame Interpolation

被引:56
|
作者
Shen, Wang [1 ]
Bao, Wenbo [1 ]
Zhai, Guangtao [1 ]
Chen, Li [1 ]
Min, Xiongkuo [1 ]
Gao, Zhiyong [1 ]
机构
[1] Shanghai Jiao Tong Univ, Inst Image Commun & Network Engn, Shanghai, Peoples R China
基金
中国国家自然科学基金;
关键词
D O I
10.1109/CVPR42600.2020.00516
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Existing works reduce motion blur and up-convert frame rate through two separate ways, including frame deblurring and frame interpolation. However, few studies have approached the joint video enhancement problem, namely synthesizing high-frame-rate clear results from low-frame-rate blurry inputs. In this paper, we propose a blurry video frame interpolation method to reduce motion blur and up-convert frame rate simultaneously. Specifically, we develop a pyramid module to cyclically synthesize clear intermediate frames. The pyramid module features adjustable spatial receptive field and temporal scope, thus contributing to controllable computational complexity and restoration ability. Besides, we propose an inter-pyramid recurrent module to connect sequential models to exploit the temporal relationship. The pyramid module integrates a recurrent module, thus can iteratively synthesize temporally smooth results without significantly increasing the model size. Extensive experimental results demonstrate that our method performs favorably against state-of-the-art methods. The source code and pre-trained model are available at https://github.com/1aomao0/BIN.
引用
收藏
页码:5113 / 5122
页数:10
相关论文
共 50 条
  • [1] Deep Inter Prediction via Reference Frame Interpolation for Blurry Video Coding
    Zhu, Zezhi
    Zhao, Lili
    Lin, Xuhu
    Guo, Xuezhou
    Chen, Jianwen
    [J]. 2021 INTERNATIONAL CONFERENCE ON VISUAL COMMUNICATIONS AND IMAGE PROCESSING (VCIP), 2021,
  • [2] Video Frame Interpolation Transformer
    Shi, Zhihao
    Xu, Xiangyu
    Liu, Xiaohong
    Chen, Jun
    Yang, Ming-Hsuan
    [J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 17461 - 17470
  • [3] PhaseNet for Video Frame Interpolation
    Meyer, Simone
    Djelouah, Abdelaziz
    McWilliams, Brian
    Sorkine-Hornung, Alexander
    Gross, Markus
    Schroers, Christopher
    [J]. 2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, : 498 - 507
  • [4] Event-based Blurry Frame Interpolation under Blind Exposure
    Weng, Wenming
    Zhang, Yueyi
    Xiong, Zhiwei
    [J]. 2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR, 2023, : 1588 - 1598
  • [5] Video Frame Interpolation with Transformer
    Lu, Liying
    Wu, Ruizheng
    Lin, Huaijia
    Lu, Jiangbo
    Jia, Jiaya
    [J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 3522 - 3532
  • [6] Softmax Splatting for Video Frame Interpolation
    Niklaus, Simon
    Liu, Feng
    [J]. 2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, : 5436 - 5445
  • [7] Exploring Discontinuity for Video Frame Interpolation
    Lee, Sangjin
    Lee, Hyeongmin
    Shin, Chajin
    Son, Hanbin
    Lee, Sangyoun
    [J]. 2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2023, : 9791 - 9800
  • [8] XVFI: eXtreme Video Frame Interpolation
    Sim, Hyeonjun
    Oh, Jihyong
    Kim, Munchurl
    [J]. 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, : 14469 - 14478
  • [9] Video Frame Interpolation: A Comprehensive Survey
    Dong, Jiong
    Ota, Kaoru
    Dong, Mianxiong
    [J]. ACM TRANSACTIONS ON MULTIMEDIA COMPUTING COMMUNICATIONS AND APPLICATIONS, 2023, 19 (02)
  • [10] Deep frame interpolation for video compression
    Begaint, Jean
    Galpin, Franck
    Guillotel, Philippe
    Guillemot, Christine
    [J]. 2019 DATA COMPRESSION CONFERENCE (DCC), 2019, : 556 - 556