Video saliency prediction for First-Person View UAV videos: Dataset and benchmark

被引:0
|
作者
Cai, Hao [1 ]
Zhang, Kao [2 ]
Chen, Zhao [1 ]
Jiang, Chenxi [1 ]
Chen, Zhenzhong [1 ]
机构
[1] Wuhan Univ, Sch Remote Sensing & Informat Engn, Wuhan 430079, Peoples R China
[2] Nanjing Univ Informat Sci & Technol, Sch Artificial Intelligence, Sch Future Technol, Nanjing 210044, Peoples R China
基金
中国博士后科学基金; 中国国家自然科学基金;
关键词
Video saliency prediction; Visual attention; First-person view; UAV; VISUAL-ATTENTION; MODEL; FIXATION; BEHAVIOR; IMAGE; GAZE;
D O I
10.1016/j.neucom.2024.127876
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Visual saliency prediction plays a crucial role in Unmanned Aerial Vehicle (UAV) video analysis tasks. In this paper, an eye -tracking dataset of the immersive viewing of videos captured from a First -Person View (FPV) of UAVs is developed, which consists of 200 video clips captured by DJI FPV drones, with a resolution of 4K QHD. The videos cover six different genres and fourteen unique scenes. To study human visual attention in watching FPV videos, fixation points are recorded using an eye tracker integrated into a VR headset. Based on the dataset, a simple yet effective FPV UAV video Saliency prediction model (FUAVSal) is proposed as a baseline, considering spatial-temporal feature, camera motion information and FPV prior. To establish benchmarks for saliency prediction in immersive FPV UAV video viewing, sixteen computational models are evaluated on this dataset. Detailed quantitative and qualitative comparisons are provided. The developed dataset and benchmarks aim to facilitate research on visual saliency prediction for First -Person View UAV videos.
引用
收藏
页数:14
相关论文
共 50 条
  • [21] A First-Person Vision Dataset of Office Activities
    Abebe, Girmaw
    Catala, Andreu
    Cavallaro, Andrea
    [J]. MULTIMODAL PATTERN RECOGNITION OF SOCIAL SIGNALS IN HUMAN-COMPUTER-INTERACTION, MPRSS 2018, 2019, 11377 : 27 - 37
  • [22] Expanding the View of First-Person Narration
    Andrea Schwenke Wyile
    [J]. Children's Literature in Education, 1999, 30 : 185 - 202
  • [23] First-Person Point-of-View Instructional Video on Lumbar Puncture Procedure
    Hatt, Danielle
    Zimmerman, Elise
    Chang, Elizabeth
    Vane, Jackson
    Hollenbach, Kathryn A.
    Shah, Ashish
    [J]. PEDIATRIC EMERGENCY CARE, 2023, 39 (12) : 953 - 956
  • [24] Future pedestrian location prediction in first-person videos for autonomous vehicles and social robots
    Chen, Kai
    Zhu, Haihua
    Tang, Dunbing
    Zheng, Kun
    [J]. IMAGE AND VISION COMPUTING, 2023, 134
  • [25] Measuring and Improving the Viewing Experience of First-person Videos
    Ma, Biao
    Reibman, Amy R.
    [J]. PROCEEDINGS OF THE THEMATIC WORKSHOPS OF ACM MULTIMEDIA 2017 (THEMATIC WORKSHOPS'17), 2017, : 493 - 501
  • [26] Unsupervised Traffic Accident Detection in First-Person Videos
    Yao, Yu
    Xu, Mingze
    Wang, Yuchen
    Crandall, David J.
    Atkins, Ella M.
    [J]. 2019 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS), 2019, : 273 - 280
  • [27] Identifying First-person Camera Wearers in Third-person Videos
    Fan, Chenyou
    Lee, Jangwon
    Xu, Mingze
    Singh, Krishna Kumar
    Lee, Yong Jae
    Crandall, David J.
    Ryoo, Michael S.
    [J]. 30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), 2017, : 4734 - 4742
  • [28] MOR-UAV: A Benchmark Dataset and Baselines for Moving Object Recognition in UAV Videos
    Mandal, Murari
    Kumar, Lav Kush
    Vipparthi, Santosh Kumar
    [J]. MM '20: PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, 2020, : 2626 - 2635
  • [29] Textile Antenna for First-Person View Goggles
    Andre, Luis
    Pinho, Pedro
    Gouveia, Carolina
    Loss, Caroline
    [J]. ELEKTRONIKA IR ELEKTROTECHNIKA, 2021, 27 (02) : 49 - 54
  • [30] First-person perspective video to enhance simulation
    Fukuta, Junaid
    Morgan, Justin
    [J]. CLINICAL TEACHER, 2018, 15 (03): : 231 - 235