Improved Robust Video Saliency Detection Based on Long-Term Spatial-Temporal Information

被引:86
|
作者
Chen, Chenglizhao [1 ]
Wang, Guotao [1 ]
Peng, Chong [1 ]
Zhang, Xiaowei [1 ]
Qin, Hong [2 ]
机构
[1] Qingdao Univ, Coll Comp Sci & Technol, Qingdao 266071, Peoples R China
[2] SUNY Stony Brook, Dept Comp Sci, Stony Brook, NY 11794 USA
基金
美国国家科学基金会; 中国国家自然科学基金;
关键词
Saliency detection; Deep learning; Quality assessment; Trajectory; Computational modeling; Color; Training; Video saliency detection; spatial-temporal saliency consistency; low-level saliency clues; long-term information revealing; OBJECT DETECTION; SEGMENTATION; OPTIMIZATION;
D O I
10.1109/TIP.2019.2934350
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This paper proposes to utilize supervised deep convolutional neural networks to take full advantage of the long-term spatial-temporal information in order to improve the video saliency detection performance. The conventional methods, which use the temporally neighbored frames solely, could easily encounter transient failure cases when the spatial-temporal saliency clues are less-trustworthy for a long period. To tackle the aforementioned limitation, we plan to identify those beyond-scope frames with trustworthy long-term saliency clues first and then align it with the current problem domain for an improved video saliency detection.
引用
收藏
页码:1090 / 1100
页数:11
相关论文
共 50 条
  • [1] Video Captioning Based on the Spatial-Temporal Saliency Tracing
    Zhou, Yuanen
    Hu, Zhenzhen
    Liu, Xueliang
    Wang, Meng
    [J]. ADVANCES IN MULTIMEDIA INFORMATION PROCESSING, PT I, 2018, 11164 : 59 - 70
  • [2] Contrast Based Hierarchical Spatial-Temporal Saliency for Video
    Le, Trung-Nghia
    Sugimoto, Akihiro
    [J]. IMAGE AND VIDEO TECHNOLOGY, PSIVT 2015, 2016, 9431 : 734 - 748
  • [3] INSHORE SHIP CHANGE DETECTION BASED ON SPATIAL-TEMPORAL SALIENCY
    Ma, Long
    Liu, Wenchao
    Han, Zhong
    Wang, Jue
    Chen, He
    [J]. 2019 IEEE INTERNATIONAL GEOSCIENCE AND REMOTE SENSING SYMPOSIUM (IGARSS 2019), 2019, : 1641 - 1644
  • [4] Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection
    Min, Kyle
    Roy, Sourya
    Tripathi, Subarna
    Guha, Tanaya
    Majumdar, Somdeb
    [J]. COMPUTER VISION - ECCV 2022, PT XXXV, 2022, 13695 : 371 - 387
  • [5] VIDEO RETARGETING WITH NONLINEAR SPATIAL-TEMPORAL SALIENCY FUSION
    Lu, Taoran
    Yuan, Zheng
    Huang, Yu
    Wu, Dapeng
    Yu, Heather
    [J]. 2010 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, 2010, : 1801 - 1804
  • [6] A video segmentation algorithm based on spatial-temporal information
    Zhu, H
    Li, ZM
    [J]. 2002 INTERNATIONAL CONFERENCE ON COMMUNICATIONS, CIRCUITS AND SYSTEMS AND WEST SINO EXPOSITION PROCEEDINGS, VOLS 1-4, 2002, : 566 - 569
  • [7] Video Saliency Prediction Based on Spatial-Temporal Two-Stream Network
    Zhang, Kao
    Chen, Zhenzhong
    [J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2019, 29 (12) : 3544 - 3557
  • [8] Spatial-Temporal Aware Long-Term Object Tracking
    Zhang, Wei
    Kang, Baosheng
    Zhang, Shunli
    [J]. IEEE ACCESS, 2020, 8 : 71662 - 71684
  • [9] A Spatial-Temporal Recurrent Neural Network for Video Saliency Prediction
    Zhang, Kao
    Chen, Zhenzhong
    Liu, Shan
    [J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, 2021, 30 : 572 - 587
  • [10] A VIDEO SALIENCY DETECTION METHOD BASED ON SPATIAL AND MOTION INFORMATION
    Xue, Kang
    Wang, Xiying
    Ma, Gengyu
    Wang, Haitao
    Nam, Dongkyung
    [J]. 2015 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2015, : 412 - 416