Improving Query and Assessment Quality in Text-Based Interactive Video Retrieval Evaluation

被引：1

作者：

Bailer, Werner ^{[1
]}

Arnold, Rahel ^{[2
]}

Benz, Vera ^{[2
]}

Coccomini, Davide Alessandro ^{[3
]}

Gkagkas, Anastasios ^{[4
]}

Gudmundsson, Gylfi Thor ^{[5
]}

Heller, Silvan ^{[2
]}

Jonsson, Bjorn Thor ^{[5
]}

Lokoc, Jakub ^{[6
]}

Messina, Nicola ^{[3
]}

Pantelidis, Nick ^{[4
]}

Wu, Jiaxin ^{[7
]}

机构：

[1] JOANNEUM Res, Graz, Austria

[2] Univ Basel, Basel, Switzerland

[3] CNR ISTI, Pisa, Italy

[4] CERTH ITI, Thessaloniki, Greece

[5] Reykjavik Univ, Reykjavik, Iceland

[6] Charles Univ Prague, Prague, Czech Republic

[7] City Univ Hong Kong, Hong Kong, Peoples R China

来源：

PROCEEDINGS OF THE 2023 ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA RETRIEVAL, ICMR 2023 | 2023年

关键词：

video retrieval; evaluation; benchmarking; quality assurance;

D O I：

10.1145/3591106.3592281

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Different task interpretations are a highly undesired element in interactive video retrieval evaluations. When a participating team focuses partially on a wrong goal, the evaluation results might become partially misleading. In this paper, we propose a process for refining known-item and open-set type queries, and preparing the assessors that judge the correctness of submissions to openset queries. Our findings from recent years reveal that a proper methodology can lead to objective query quality improvements and subjective participant satisfaction with query clarity.

引用

页码：597 / 601

页数：5

共 50 条

[41] DotCHA: An Interactive 3D Text-based CAPTCHA
Kim, Suzi
Choi, Sunghee
JOURNAL OF WEB ENGINEERING, 2020, 18 (08): : 837 - 863
[42] A Text-Based Deep Reinforcement Learning Framework for Interactive Recommendation
Wang, Chaoyang
Guo, Zhiqiang
Li, Jianjun
Pan, Peng
Li, Guohui
ECAI 2020: 24TH EUROPEAN CONFERENCE ON ARTIFICIAL INTELLIGENCE, 2020, 325 : 537 - 544
[43] Old fashion text-based image retrieval using FCA
Ahmad, I
Jang, TS
2003 INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, VOL 3, PROCEEDINGS, 2003, : 33 - 36
[44] Learning Semantic Polymorphic Mapping for Text-Based Person Retrieval
Li J.
Jiang M.
Kong J.
Tao X.
Luo X.
IEEE Transactions on Multimedia, 2024, 26 : 1 - 14
[45] Adaptive Uncertainty-Based Learning for Text-Based Person Retrieval
Li, Shenshen
He, Chen
Xu, Xing
Shen, Fumin
Yang, Yang
Shen, Heng Tao
THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 4, 2024, : 3172 - 3180
[46] Exploiting Visual Concepts to Improve Text-Based Image Retrieval
Tollari, Sabrina
Detyniecki, Marcin
Marsala, Christophe
Fakeri-Tabrizi, Ali
Amini, Massih-Reza
Gallinari, Patrick
ADVANCES IN INFORMATION RETRIEVAL, PROCEEDINGS, 2009, 5478 : 701 - 705
[47] SUM: Serialized Updating and Matching for text-based person retrieval
Wang, Zijie
Zhu, Aichun
Xue, Jingyi
Jiang, Daihong
Liu, Chao
Li, Yifeng
Hu, Fangqiang
KNOWLEDGE-BASED SYSTEMS, 2022, 248
[48] Text-Based and Content-Based Image Retrieval on Flickr: DEMO
Manuel Barrios, Juan
Diaz-Espinoza, Diego
Bustos, Benjamin
SISAP 2009: 2009 SECOND INTERNATIONAL WORKSHOP ON SIMILARITY SEARCH AND APPLICATIONS, PROCEEDINGS, 2009, : 156 - 157
[49] Simulating Virtual Organizations for Research: A Comparative Empirical Evaluation of Text-Based, Video, and Virtual Reality Video Vignettes
van Zelderen, Anand P. A.
Masters-Waage, Theodore C.
Dries, Nicky
Menges, Jochen I.
Sanchez, Diana R.
ORGANIZATIONAL RESEARCH METHODS, 2024,
[50] Document Expansion for Text-Based Image Retrieval at CLEF 2009
Min, Jinming
Wilkins, Peter
Leveling, Johannes
Jones, Gareth J. F.
MULTILINGUAL INFORMATION ACCESS EVALUATION II: MULTIMEDIA EXPERIMENTS, PT II, 2010, 6242 : 172 - 176

← 1 2 3 4 5 →