Temporally Consistent Semantic Video Editing

被引:7
|
作者
Xu, Yiran [1 ]
AlBahar, Badour [2 ]
Huang, Jia-Bin [1 ]
机构
[1] Univ Maryland, College Pk, MD 20742 USA
[2] Virginia Tech, Blacksburg, VA USA
来源
关键词
Video editing; GAN editing; Video consistency;
D O I
10.1007/978-3-031-19784-0_21
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Generative adversarial networks (GANs) have demonstrated impressive image generation quality and semantic editing capability of real images, e.g., changing object classes, modifying attributes, or transferring styles. However, applying these GAN-based editing to a video independently for each frame inevitably results in temporal flickering artifacts. We present a simple yet effective method to facilitate temporally coherent video editing. Our core idea is to minimize the temporal photometric inconsistency by optimizing both the latent code and the pre-trained generator. We evaluate the quality of our editing on different domains and GAN inversion techniques and show favorable results against the baselines.
引用
收藏
页码:357 / 374
页数:18
相关论文
共 50 条
  • [41] Towards Consistent Video Editing with Text-to-Image Diffusion Models
    Zhang, Zicheng
    Li, Bonan
    Nie, Xuecheng
    Han, Congying
    Guo, Tiande
    Liu, Luoqi
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [42] Context-Consistent Semantic Image Editing with Style-Preserved Modulation
    Luo, Wuyang
    Yang, Su
    Wang, Hong
    Long, Bo
    Zhang, Weishan
    COMPUTER VISION - ECCV 2022, PT XVII, 2022, 13677 : 561 - 578
  • [43] Temporally consistent video colorization with deep feature propagation and self-regularization learning
    Yihao Liu
    Hengyuan Zhao
    Kelvin C. K. Chan
    Xintao Wang
    Chen Change Loy
    Yu Qiao
    Chao Dong
    Computational Visual Media, 2024, 10 : 375 - 395
  • [44] Temporally Consistent Depth Map Estimation for 3D Video Generation and Coding
    Lee, Sang-Beom
    Ho, Yo-Sung
    CHINA COMMUNICATIONS, 2013, 10 (05) : 39 - 49
  • [45] Spatio-temporally Consistent Multi-view Video Synthesis for Autostereoscopic Displays
    Lin, Shu-Jyuan
    Cheng, Chia-Ming
    Lai, Shang-Hong
    ADVANCES IN MULTIMEDIA INFORMATION PROCESSING - PCM 2009, 2009, 5879 : 532 - 542
  • [46] Temporally consistent reconstruction from multiple video streams using enhanced belief propagation
    Larsen, E. Scott
    Mordohai, Philippos
    Pollefeys, Marc
    Fuchs, Henry
    2007 IEEE 11TH INTERNATIONAL CONFERENCE ON COMPUTER VISION, VOLS 1-6, 2007, : 1440 - 1447
  • [47] Spatio-Temporally Consistent Color and Structure Optimization for Multiview Video Color Correction
    Lu, Shao-Ping
    Ceulemans, Beerend
    Munteanu, Adrian
    Schelkens, Peter
    IEEE TRANSACTIONS ON MULTIMEDIA, 2015, 17 (05) : 577 - 590
  • [48] Temporally consistent video colorization with deep feature propagation and self-regularization learning
    Liu, Yihao
    Zhao, Hengyuan
    Chan, Kelvin C. K.
    Wang, Xintao
    Loy, Chen Change
    Qiao, Yu
    Dong, Chao
    COMPUTATIONAL VISUAL MEDIA, 2024, 10 (02) : 375 - 395
  • [49] Joint 2D-3D Temporally Consistent Semantic Segmentation of Street Scenes
    Floros, Georgios
    Leibe, Bastian
    2012 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2012, : 2823 - 2830
  • [50] Temporally Consistent Horizon Lines
    Kluger, Florian
    Ackermann, Hanno
    Yang, Michael Ying
    Rosenhahn, Bodo
    2020 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), 2020, : 3161 - 3167