Multi-Task Learning Framework for Motion Estimation and Dynamic Scene Deblurring

被引:3
|
作者
Jung, Hyungjoo [1 ]
Kim, Youngjung [2 ]
Jang, Hyunsung [3 ,4 ]
Ha, Namkoo [3 ]
Sohn, Kwanghoon [4 ]
机构
[1] Korea Inst Sci & Technol, Artificial Intelligence & Robot Inst, Seoul 02792, South Korea
[2] Agcy Def Dev, Inst Def Adv Technol Res, Daejeon 34060, South Korea
[3] LIG Nex1 Co Ltd, EO IR Res & Dev Lab, Yongin 16911, South Korea
[4] Yonsei Univ, Sch Elect & Elect Engn, Seoul 120749, South Korea
基金
新加坡国家研究基金会;
关键词
Kernel; Dynamics; Task analysis; Cameras; Motion estimation; Image restoration; Estimation; Motion blur; dynamic scene deblurring; motion estimation; multi-task learning; KERNEL ESTIMATION; DEPTH ESTIMATION;
D O I
10.1109/TIP.2021.3113185
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Motion blur, which disturbs human and machine perceptions of a scene, has been considered an unnecessary artifact that should be removed. However, the blur can be a useful clue to understanding the dynamic scene, since various sources of motion generate different types of artifacts. Motivated by the relationship between motion and blur, we propose a motion-aware feature learning framework for dynamic scene deblurring through multi-task learning. Our multi-task framework simultaneously estimates a deblurred image and a motion field from a blurred image. We design the encoder-decoder architectures for two tasks, and the encoder part is shared between them. Our motion estimation network could effectively distinguish between different types of blur, which facilitates image deblurring. Understanding implicit motion information through image deblurring could improve the performance of motion estimation. In addition to sharing the network between two tasks, we propose a reblurring loss function to optimize the overall parameters in our multi-task architecture. We provide an intensive analysis of complementary tasks to show the effectiveness of our multi-task framework. Furthermore, the experimental results demonstrate that the proposed method outperforms the state-of-the-art deblurring methods with respect to both qualitative and quantitative evaluations.
引用
收藏
页码:8170 / 8183
页数:14
相关论文
共 50 条
  • [1] A Multi-Task Learning Framework for Head Pose Estimation under Target Motion
    Yan, Yan
    Ricci, Elisa
    Subramanian, Ramanathan
    Liu, Gaowen
    Lanz, Oswald
    Sebe, Nicu
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2016, 38 (06) : 1070 - 1083
  • [2] A multi-task learning framework for gas detection and concentration estimation
    Liu, Huixiang
    Li, Qing
    Gu, Yu
    [J]. NEUROCOMPUTING, 2020, 416 : 28 - 37
  • [3] Joint Learning of Image Deblurring and Depth Estimation Through Adversarial Multi-Task Network
    Hou, Shengyu
    Fu, Mengyin
    Song, Wenjie
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2023, 33 (12) : 7327 - 7341
  • [4] Impact of Acoustic Event Tagging on Scene Classification in a Multi-Task Learning Framework
    Parikh, Rahil
    Sundar, Harshavardhan
    Sun, Ming
    Wang, Chao
    Matsoukas, Spyros
    [J]. INTERSPEECH 2022, 2022, : 4192 - 4196
  • [5] Multi-Task Collaboration Deep Learning Framework for Infrared Precipitation Estimation
    Yang, Xuying
    Sun, Peng
    Zhang, Feng
    Du, Zhenhong
    Liu, Renyi
    [J]. REMOTE SENSING, 2021, 13 (12)
  • [6] Multi-view representation learning in multi-task scene
    Run-kun Lu
    Jian-wei Liu
    Si-ming Lian
    Xin Zuo
    [J]. Neural Computing and Applications, 2020, 32 : 10403 - 10422
  • [7] A multi-task approach to face deblurring
    Ziyi Shen
    Tingfa Xu
    Jizhou Zhang
    Jie Guo
    Shenwang Jiang
    [J]. EURASIP Journal on Wireless Communications and Networking, 2019
  • [8] Multi-view representation learning in multi-task scene
    Lu, Run-kun
    Liu, Jian-wei
    Lian, Si-ming
    Zuo, Xin
    [J]. NEURAL COMPUTING & APPLICATIONS, 2020, 32 (14): : 10403 - 10422
  • [9] A multi-task approach to face deblurring
    Shen, Ziyi
    Xu, Tingfa
    Zhang, Jizhou
    Guo, Jie
    Jiang, Shenwang
    [J]. EURASIP JOURNAL ON WIRELESS COMMUNICATIONS AND NETWORKING, 2019, 2019 (1)
  • [10] Novel Multi-Task Learning for Motion Magnification
    Chen, Li
    Peng, Cong
    Zhao, Bingchao
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2023, 33 (10) : 5973 - 5985