OpenSearch: Lessons Learned from an Online Evaluation Campaign

被引：11

作者：

Jagerman, Rolf ^{[1
]}

Balog, Krisztian ^{[2
]}

De Rijke, Maarten ^{[1
]}

机构：

[1] Univ Amsterdam, Sci Pk 904, NL-1098 XH Amsterdam, Netherlands

[2] Univ Stavanger, Kristine Bonnevies Vei 22, N-4021 Stavanger, Norway

来源：

ACM JOURNAL OF DATA AND INFORMATION QUALITY | 2018年 / 10卷 / 03期

关键词：

Living labs; online evaluation;

D O I：

10.1145/3239575

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

We report on our experience with TREC OpenSearch, an online evaluation campaign that enabled researchers to evaluate their experimental retrieval methods using real users of a live website. Specifically, we focus on the task of ad hoc document retrieval within the academic search domain, and work with two search engines, CiteSeerX and SSOAR, that provide us with traffic. We describe our experimental platform, which is based on the living labs methodology, and report on the experimental results obtained. We also share our experiences, challenges, and the lessons learned from running this track in 2016 and 2017.

引用

页数：15

共 50 条

[31] Lessons learned: Online analyzers in utilities
Huchler, Loraine A.
HYDROCARBON PROCESSING, 2008, 87 (02): : 138 - 138
[32] COUNTERING ONLINE RADICALIZATION - LESSONS LEARNED
Stefanescu, Daniela
Stefan, Teodoru
REDEFINING COMMUNITY IN INTERCULTURAL CONTEXT, RCIC' 18, 2018, 7 (01): : 385 - 390
[33] Online Interprofessional Education - Lessons Learned
Callard, Pat
Tegzes, John
Mackintosh, Susan
JOURNAL OF INTERPROFESSIONAL CARE, 2013, 27 : 167 - 167
[34] Inclusive Teaching Online: Lessons Learned
Benson, Sarah K.
Alborno, Nadera
INTERNATIONAL JOURNAL OF DISABILITY DEVELOPMENT AND EDUCATION, 2024,
[35] Crowdfunding Astronomy Outreach Projects: Lessons learned from the UNAWE crowdfunding campaign
Ashton, Abi J.
Russo, Pedro
Heenatigala, Thilina
CAPJOURNAL, 2014, (16): : 24 - 27
[36] Research Collaboration in a Communication Rights Campaign: Lessons Learned
Ryan, Charlotte
NEW SOLUTIONS-A JOURNAL OF ENVIRONMENTAL AND OCCUPATIONAL HEALTH POLICY, 2018, 28 (02) : 303 - 320
[37] Lessons learned from the evaluation of Spanish Language Models
Agerri, Rodrigo
Agirre, Eneko
PROCESAMIENTO DEL LENGUAJE NATURAL, 2023, (70): : 157 - 170
[38] The Evaluation of Information Systems: Lessons Learned from Practice
Perego, Angela
MANAGEMENT OF THE INTERCONNECTED WORLD, 2010, : 325 - 332
[39] Lessons learned from an evaluation of referrals to the emergency department
Gorodetzer, Roee
Alpert, Evan Avraham
Orr, Zvika
Unger, Shifra
Zalut, Todd
ISRAEL JOURNAL OF HEALTH POLICY RESEARCH, 2020, 9 (01)
[40] LESSONS LEARNED TO DATE FROM THE SHIPPINGPORT AGING EVALUATION
ALLEN, RP
JOHNSON, AB
NUCLEAR ENGINEERING AND DESIGN, 1990, 118 (03) : 319 - 327

← 1 2 3 4 5 →