Temporally Consistent Semantic Video Editing

被引:7
|
作者
Xu, Yiran [1 ]
AlBahar, Badour [2 ]
Huang, Jia-Bin [1 ]
机构
[1] Univ Maryland, College Pk, MD 20742 USA
[2] Virginia Tech, Blacksburg, VA USA
来源
关键词
Video editing; GAN editing; Video consistency;
D O I
10.1007/978-3-031-19784-0_21
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Generative adversarial networks (GANs) have demonstrated impressive image generation quality and semantic editing capability of real images, e.g., changing object classes, modifying attributes, or transferring styles. However, applying these GAN-based editing to a video independently for each frame inevitably results in temporal flickering artifacts. We present a simple yet effective method to facilitate temporally coherent video editing. Our core idea is to minimize the temporal photometric inconsistency by optimizing both the latent code and the pre-trained generator. We evaluate the quality of our editing on different domains and GAN inversion techniques and show favorable results against the baselines.
引用
收藏
页码:357 / 374
页数:18
相关论文
共 50 条
  • [1] UniFaceGAN: A Unified Framework for Temporally Consistent Facial Video Editing
    Cao, Meng
    Huang, Haozhi
    Wang, Hao
    Wang, Xuan
    Shen, Li
    Wang, Sheng
    Bao, Linchao
    Li, Zhifeng
    Luo, Jiebo
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 : 6107 - 6116
  • [2] Temporally consistent gaussian random field for video semantic analysis
    Tang, Jinhui
    Hua, Xian-Sheng
    Mei, Tao
    Qi, Guo-Jun
    Li, Shipeng
    Wu, Xiuqing
    2007 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, VOLS 1-7, 2007, : 2221 - +
  • [3] Diffusion Video Autoencoders: Toward Temporally Consistent Face Video Editing via Disentangled Video Encoding
    Kim, Gyeongman
    Shim, Hajin
    Kim, Hyunsu
    Choi, Yunjey
    Kim, Junho
    Yang, Eunho
    2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR, 2023, : 6091 - 6100
  • [4] Complete and temporally consistent video outpainting
    Dehan, Loic
    Van Ranst, Wiebe
    Vandewalle, Patrick
    Goedeme, Toon
    2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2022, 2022, : 686 - 694
  • [5] AuxAdapt: Stable and Efficient Test-Time Adaptation for Temporally Consistent Video Semantic Segmentation
    Zhang, Yizhe
    Borse, Shubhankar
    Cai, Hong
    Porikli, Fatih
    2022 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2022), 2022, : 2633 - 2642
  • [6] A motional but temporally consistent physical video examples
    Du, Zhenyu
    Wei, Xingxing
    Zhang, Weiming
    Liu, Fangzheng
    Bian, Huanyu
    Liu, Jiayang
    JOURNAL OF INFORMATION SECURITY AND APPLICATIONS, 2022, 69
  • [7] TempFormer: Temporally Consistent Transformer for Video Denoising
    Song, Mingyang
    Zhang, Yang
    Aydin, Tunc O.
    COMPUTER VISION, ECCV 2022, PT XIX, 2022, 13679 : 481 - 496
  • [8] A motional but temporally consistent physical video examples
    Du, Zhenyu
    Wei, Xingxing
    Zhang, Weiming
    Liu, Fangzheng
    Bian, Huanyu
    Liu, Jiayang
    Journal of Information Security and Applications, 2022, 69
  • [9] Towards Spatio-Temporally Consistent Semantic Mapping
    Zhao, Zhe
    Chen, Xiaoping
    ROBOCUP 2014: ROBOT WORLD CUP XVIII, 2015, 8992 : 258 - 269
  • [10] VIDEO STEREO MATCHING WITH TEMPORALLY CONSISTENT BELIEF PROPAGATION
    Hou, Hsin-Yu
    Wu, Sih-Sian
    Chang, Da-Fang
    Chen, Liang-Gee
    2018 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO (ICME), 2018,