Video saliency prediction for First-Person View UAV videos: Dataset and benchmark

被引:0
|
作者
Cai, Hao [1 ]
Zhang, Kao [2 ]
Chen, Zhao [1 ]
Jiang, Chenxi [1 ]
Chen, Zhenzhong [1 ]
机构
[1] Wuhan Univ, Sch Remote Sensing & Informat Engn, Wuhan 430079, Peoples R China
[2] Nanjing Univ Informat Sci & Technol, Sch Artificial Intelligence, Sch Future Technol, Nanjing 210044, Peoples R China
基金
中国博士后科学基金; 中国国家自然科学基金;
关键词
Video saliency prediction; Visual attention; First-person view; UAV; VISUAL-ATTENTION; MODEL; FIXATION; BEHAVIOR; IMAGE; GAZE;
D O I
10.1016/j.neucom.2024.127876
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Visual saliency prediction plays a crucial role in Unmanned Aerial Vehicle (UAV) video analysis tasks. In this paper, an eye -tracking dataset of the immersive viewing of videos captured from a First -Person View (FPV) of UAVs is developed, which consists of 200 video clips captured by DJI FPV drones, with a resolution of 4K QHD. The videos cover six different genres and fourteen unique scenes. To study human visual attention in watching FPV videos, fixation points are recorded using an eye tracker integrated into a VR headset. Based on the dataset, a simple yet effective FPV UAV video Saliency prediction model (FUAVSal) is proposed as a baseline, considering spatial-temporal feature, camera motion information and FPV prior. To establish benchmarks for saliency prediction in immersive FPV UAV video viewing, sixteen computational models are evaluated on this dataset. Detailed quantitative and qualitative comparisons are provided. The developed dataset and benchmarks aim to facilitate research on visual saliency prediction for First -Person View UAV videos.
引用
收藏
页数:14
相关论文
共 50 条
  • [1] Supervised saliency maps for first-person videos based on sparse coding
    Li, Yujie
    Kanemura, Atsunori
    Asoh, Hideki
    Miyanishi, Taiki
    Kawanabe, Motoaki
    [J]. 2018 ASIA-PACIFIC SIGNAL AND INFORMATION PROCESSING ASSOCIATION ANNUAL SUMMIT AND CONFERENCE (APSIPA ASC), 2018, : 2000 - 2005
  • [2] Supervised Saliency Mapping for First-Person Videos With an Inverse Sparse Coding Framework
    Li, Yujie
    Akaho, Shotaro
    Asoh, Hideki
    Tan, Benying
    [J]. IEEE ACCESS, 2019, 7 : 12547 - 12556
  • [3] Future Person Localization in First-Person Videos
    Yagi, Takuma
    Mangalam, Karttikeya
    Yonetani, Ryo
    Sato, Yoichi
    [J]. 2018 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2018, : 7593 - 7602
  • [4] AR Tips: Augmented First-Person View Task Instruction Videos
    Lee, Gun A.
    Ahn, Seungjun
    Hoff, William
    Billinghurst, Mark
    [J]. ADJUNCT PROCEEDINGS OF THE 2019 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR-ADJUNCT 2019), 2019, : 34 - 36
  • [5] Assessing the impact of autonomy and overconfidence in UAV first-person view training
    Li, Songpo
    Cummings, Mary L.
    Welton, Benjamin
    [J]. Applied Ergonomics, 2022, 98
  • [6] CHARACTERIZING DISTORTIONS IN FIRST-PERSON VIDEOS
    Bai, Chen
    Reibman, Amy R.
    [J]. 2016 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING (ICIP), 2016, : 2440 - 2444
  • [7] Assessing the impact of autonomy and overconfidence in UAV first-person view training
    Li, Songpo
    Cummings, Mary L.
    Welton, Benjamin
    [J]. APPLIED ERGONOMICS, 2022, 98
  • [8] Vibrotactile Rendering of Camera Motion for Bimanual Experience of First-Person View Videos
    Gongora, Daniel
    Nagano, Hikaru
    Konyo, Masashi
    Tadokoro, Satoshi
    [J]. 2017 IEEE WORLD HAPTICS CONFERENCE (WHC), 2017, : 454 - 459
  • [9] Enhancing First-Person View Task Instruction Videos with Augmented Reality Cues
    Lee, Gun A.
    Ahn, Seungjun
    Hoff, William
    Billinghurst, Mark
    [J]. 2020 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR 2020), 2020, : 498 - 508
  • [10] T2FPV: Dataset and Method for Correcting First-Person View Errors in Pedestrian Trajectory Prediction
    Stoler, Benjamin
    Jana, Meghdeep
    Hwang, Soonmin
    Oh, Jean
    [J]. 2023 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS, IROS, 2023, : 4037 - 4044