Multi-modal Interactive Video Retrieval with Temporal Queries

被引：10

作者：

Heller, Silvan ^{[1
]}

Arnold, Rahel ^{[1
]}

Gasser, Ralph ^{[1
]}

Gsteiger, Viktor ^{[1
]}

Parian-Scherb, Mahnaz ^{[1
]}

Rossetto, Luca ^{[2
]}

Sauter, Loris ^{[1
]}

Spiess, Florian ^{[1
]}

Schuldt, Heiko ^{[1
]}

机构：

[1] Univ Basel, Dept Math & Comp Sci, Basel, Switzerland

[2] Univ Zurich, Dept Informat, Zurich, Switzerland

来源：

MULTIMEDIA MODELING, MMM 2022, PT II | 2022年 / 13142卷

基金：

瑞士国家科学基金会;

关键词：

Video Browser Showdown; Interactive video retrieval; Content-based retrieval;

D O I：

10.1007/978-3-030-98355-0_44

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

This paper presents the version of vitrivr participating at the Video Browser Showdown (VBS) 2022. vitrivr already supports a wide range of query modalities, such as color and semantic sketches, OCR, ASR and text embedding. In this paper, we briefly introduce the system, then describe our new approach to queries specifying temporal context, ideas for color-based sketches in a competitive retrieval setting and a novel approach to pose-based queries.

引用

页码：493 / 498

页数：6

共 50 条

[1] An Interactive Video Search Platform for Multi-modal Retrieval with Advanced Concepts
Nguyen-Khang Le
Dieu-Hien Nguyen
Minh-Triet Tran
[J]. MULTIMEDIA MODELING (MMM 2020), PT II, 2020, 11962 : 766 - 771
[2] End-to-end Knowledge Retrieval with Multi-modal Queries
Luo, Man
Fang, Zhiyuan
Gokhale, Tejas
Yang, Yezhou
Baral, Chitta
[J]. PROCEEDINGS OF THE 61ST ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2023): LONG PAPERS, VOL 1, 2023, : 8573 - 8589
[3] HUMOR: a HUman MOtion retrieval system with multi-modal queries
Wu, MY
Wu, YC
Chiu, CY
Chao, SP
Yang, SN
[J]. 2004 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXP (ICME), VOLS 1-3, 2004, : 315 - 316
[4] Multi-modal Language Models for Lecture Video Retrieval
Chen, Huizhong
Cooper, Matthew
Joshi, Dhiraj
Girod, Bernd
[J]. PROCEEDINGS OF THE 2014 ACM CONFERENCE ON MULTIMEDIA (MM'14), 2014, : 1081 - 1084
[5] Personalized Multi-modal Video Retrieval on Mobile Devices
Zhang, Haotian
Jepson, Allan D.
Mohomed, Iqbal
Derpanis, Konstantinos G.
Zhang, Ran
Fazly, Afsaneh
[J]. PROCEEDINGS OF THE 29TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2021, 2021, : 1185 - 1191
[6] A multi-modal system for the retrieval of semantic video events
Amir, A
Basu, S
Iyengar, G
Lin, CY
Naphade, M
Smith, JR
Srinivasan, S
Tseng, B
[J]. COMPUTER VISION AND IMAGE UNDERSTANDING, 2004, 96 (02) : 216 - 236
[7] Everything at Once - Multi-modal Fusion Transformer for Video Retrieval
Shvetsova, Nina
Chen, Brian
Rouditchenko, Andrew
Thomas, Samuel
Kingsbury, Brian
Feris, Rogerio
Harwath, David
Glass, James
Kuehne, Hilde
[J]. 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2022), 2022, : 19988 - 19997
[8] Multi-Modal Relational Graph for Cross-Modal Video Moment Retrieval
Zeng, Yawen
Cao, Da
Wei, Xiaochi
Liu, Meng
Zhao, Zhou
Qin, Zheng
[J]. 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021, 2021, : 2215 - 2224
[9] Dig into Multi-modal Cues for Video Retrieval with Hierarchical Alignment
Wang, Wenzhe
Zhang, Mengdan
Chen, Runnan
Cai, Guanyu
Zhou, Penghao
Peng, Pai
Guo, Xiaowei
Wu, Jian
Sun, Xing
[J]. PROCEEDINGS OF THE THIRTIETH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2021, 2021, : 1113 - 1121
[10] Deep Video Understanding with a Unified Multi-Modal Retrieval Framework
Xie, Chen-Wei
Sun, Siyang
Zhao, Liming
Wu, Jianmin
Li, Dangwei
Zheng, Yun
[J]. PROCEEDINGS OF THE 30TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2022, 2022, : 7055 - 7059

← 1 2 3 4 5 →