A Framework of Evaluation for Question-Answering Systems

被引：0
|
作者：
El Ayari, Sarra [1 ]
Grau, Brigitte [1 ]
机构：
[1] CNRS, LIMSI, F-75700 Paris, France
来源：
ADVANCES IN INFORMATION RETRIEVAL, PROCEEDINGS | 2009年 / 5478卷
关键词：
Glass-box evaluation; question-answering system; relational database; framework; relevance of criteria;
D O I：
暂无
中图分类号：
TP [自动化技术、计算机技术];
学科分类号：
0812 ;
摘要：
Evaluating complex system is a complex task. Evaluation campaigns are organized each year to test different systems on global results, but they do not evaluate the relevance of the criteria used. Our purpose consist in modifying the intermediate results created by the components and inserting the new results into the process, without modifying the components. We will describe our framework of glass-box evaluation.
引用
收藏
页码：744 / 748
页数：5
相关论文
共 50 条
  • [31] Deep Learning Powered Question-Answering Framework for Organizations Digital Transformation
    Carvalho, Nuno Ramos
    Barbosa, Luis Soares
    [J]. PROCEEDINGS OF THE 12TH INTERNATIONAL CONFERENCE ON THEORY AND PRACTICE OF ELECTRONIC GOVERNANCE (ICEGOV2019), 2019, : 76 - 79
  • [32] Multilingual Question-Answering System in Biomedical Domain on the Web: An Evaluation
    Olvera-Lobo, Maria-Dolores
    Gutierrez-Artacho, Juncal
    [J]. MULTILINGUAL AND MULTIMODAL INFORMATION ACCESS EVALUATION, 2011, 6941 : 83 - +
  • [33] FRAME-BASED INTERFACE FOR QUESTION-ANSWERING SYSTEMS.
    Takagi, Toshihisa
    Matsuo, Fumihiro
    Ushijima, Kazuo
    [J]. Proceedings - IEEE Computer Society's International Computer Software & Applications Conference, 1985, : 388 - 393
  • [34] FACT RETRIEVAL + DEDUCTIVE QUESTION-ANSWERING INFORMATION RETRIEVAL SYSTEMS
    COOPER, WS
    [J]. JOURNAL OF THE ACM, 1964, 11 (02) : 117 - &
  • [35] Framework for Question-Answering in Sanskrit through Automated Construction of Knowledge Graphs
    Terdalkar, Hrishikesh
    Bhattacharya, Arnab
    [J]. PROCEEDINGS OF THE 6TH INTERNATIONAL SANSKRIT COMPUTATIONAL LINGUISTICS SYMPOSIUM (ISCLS 2019), 2019, : 98 - 117
  • [36] A question-answering system using argumentation
    Moreale, E
    Vargas-Vera, M
    [J]. MICAI 2004: ADVANCES IN ARTIFICIAL INTELLIGENCE, 2004, 2972 : 400 - 409
  • [37] Conditional Generation with a Question-Answering Blueprint
    Narayan, Shashi
    Maynez, Joshua
    Amplayo, Reinald Kim
    Ganchev, Kuzman
    Louis, Annie
    Huot, Fantine
    Sandholm, Anders
    Das, Dipanjan
    Lapata, Mirella
    [J]. TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, 2023, 11 : 974 - 996
  • [38] A GRAMMAR BASE QUESTION-ANSWERING PROCEDURE
    ROSENBAUM, PS
    [J]. COMMUNICATIONS OF THE ACM, 1967, 10 (10) : 630 - +
  • [39] Disambiguation for Arabic Question-Answering System
    Dardour, Sondes
    Fehri, Hela
    Haddar, Kais
    [J]. FORMALIZING NATURAL LANGUAGES WITH NOOJ 2019 AND ITS NATURAL LANGUAGE PROCESSING APPLICATIONS, NOOJ 2019, 2020, 1153 : 101 - 111
  • [40] MemoriQA: A Question-Answering Lifelog Dataset
    Tran, Quang-Linh
    Nguyen, Binh
    Jones, Gareth J. F.
    Gurrin, Cathal
    [J]. PROCEEDINGS OF THE FIRST ACM WORKSHOP ON AI-POWERED QUESTION ANSWERING SYSTEMS FOR MULTIMEDIA, AIQAM 2024, 2024, : 7 - 12
12345