OpenSearch: Lessons Learned from an Online Evaluation Campaign

被引：11

作者：

Jagerman, Rolf ^{[1
]}

Balog, Krisztian ^{[2
]}

De Rijke, Maarten ^{[1
]}

机构：

[1] Univ Amsterdam, Sci Pk 904, NL-1098 XH Amsterdam, Netherlands

[2] Univ Stavanger, Kristine Bonnevies Vei 22, N-4021 Stavanger, Norway

来源：

ACM JOURNAL OF DATA AND INFORMATION QUALITY | 2018年 / 10卷 / 03期

关键词：

Living labs; online evaluation;

D O I：

10.1145/3239575

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

We report on our experience with TREC OpenSearch, an online evaluation campaign that enabled researchers to evaluate their experimental retrieval methods using real users of a live website. Specifically, we focus on the task of ad hoc document retrieval within the academic search domain, and work with two search engines, CiteSeerX and SSOAR, that provide us with traffic. We describe our experimental platform, which is based on the living labs methodology, and report on the experimental results obtained. We also share our experiences, challenges, and the lessons learned from running this track in 2016 and 2017.

引用

页数：15

共 50 条

[21] Lessons learned: Campaign advertising in the 2000 elections
Goldstein, K
Freedman, P
POLITICAL COMMUNICATION, 2002, 19 (01) : 5 - 28
[22] The 1979 Chinese campaign in Vietnam: Lessons learned
O'Dowd, EC
Corbett, JF
LESSONS OF HISTORY: THE CHINESE PEOPLE'S LIBERATION ARMY AT 75, 2003, : 353 - 378
[23] Lessons Learned from Lessons Learned
Rhodes, Lucy
Dawson, Ray
KNOWLEDGE AND PROCESS MANAGEMENT, 2013, 20 (03) : 154 - 160
[24] Evaluation and Lessons Learned From the Development and Implementation of an Online Prenatal Nutrition Training for EFNEP Paraprofessionals
Chlipalski, Micheline
Baker, Susan
Olson, Beth
Auld, Garry
JOURNAL OF NUTRITION EDUCATION AND BEHAVIOR, 2019, 51 (06) : 749 - 757
[25] Lessons learned from the directed evaluation experience
Gredler, ME
Johnson, RL
AMERICAN JOURNAL OF EVALUATION, 2001, 22 (01) : 97 - 104
[26] LESSONS LEARNED FROM THE EVALUATION OF THE HAIL INITIATIVE
Cho, J.
Thorud, J.
GERONTOLOGIST, 2015, 55 : 340 - 340
[27] Lessons Learned from Creating a Balanced Corpus from Online Data
Dargis, Roberts
Levane-Petrova, Kristine
Poikans, Ilmars
HUMAN LANGUAGE TECHNOLOGIES - THE BALTIC PERSPECTIVE (HLT 2020), 2020, 328 : 127 - 134
[28] Lessons learned from implementing remotely invigilated online exams
Cramp, Joshua
Medlin, John F.
Lake, Phoebe
Sharp, Colin
JOURNAL OF UNIVERSITY TEACHING AND LEARNING PRACTICE, 2019, 16 (01):
[29] Lessons Learned From an Online Sexual Health Education Program
McKay, Cathy
Hartzler-Weakley, Kim
Bartley, Hayden
RESEARCH QUARTERLY FOR EXERCISE AND SPORT, 2022, 93 : A98 - A99
[30] Lessons learned from teaching an online chemical safety course
Foster, BL
ABSTRACTS OF PAPERS OF THE AMERICAN CHEMICAL SOCIETY, 2005, 229 : U339 - U339

← 1 2 3 4 5 →