Evaluation in the context of natural language generation

被引：22
|
作者：
Mellish, C [1 ]
Dale, R
机构：
[1] Univ Edinburgh, Dept Artificial Intelligence, Edinburgh EH8 9YL, Midlothian, Scotland
[2] Macquarie Univ, Microsoft Res Inst, N Ryde, NSW, Australia
来源：
COMPUTER SPEECH AND LANGUAGE | 1998年 / 12卷 / 04期
关键词：
D O I：
10.1006/csla.1998.0106
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
What role should evaluation play in the development of natural language generation (NLG) techniques and systems? In this paper we describe what is involved in natural language generation, and survey how evaluation has figured in work in this area to date. We comment on the issues raised by this existing work and on how the problems of NLG evaluation are different from the problems of evaluating work in natural language understanding. The paper is concluded by suggesting a way forward by looking more closely at the component problems that are addressed in natural language generation research; a particular text generation application is examined and the issues that are raised in assessing its performance on a variety of dimensions are looked at. (C) 1998 Academic Press.
引用
收藏
页码：349 / 373
页数：25
相关论文
共 50 条
  • [1] Natural Language Generation in the context of the Semantic Web
    Bouayad-Agha, Nadjet
    Casamayor, Gerard
    Wanner, Leo
    [J]. SEMANTIC WEB, 2014, 5 (06) : 493 - 513
  • [2] Formal and computational models of context for natural language generation
    van Deemter, K
    Odijk, J
    [J]. FORMAL ASPECTS OF CONTEXT, 2000, 20 : 1 - 21
  • [3] An Overview of Natural Language Generation Systems Evaluation
    Yang, Feng-Jen
    [J]. WORLD CONGRESS ON ENGINEERING AND COMPUTER SCIENCE, WCECS 2015, VOL I, 2015, : 71 - 74
  • [4] Evaluating the Evaluation of Diversity in Natural Language Generation
    Tevet, Guy
    Berant, Jonathan
    [J]. 16TH CONFERENCE OF THE EUROPEAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EACL 2021), 2021, : 326 - 346
  • [5] Natural Language Generation, its Evaluation and Metrics
    Gehrmann, Sebastian
    Adewumi, Tosin
    Aggarwal, Karmanya
    Ammanamanchi, Pawan Sasanka
    Anuoluwapo, Aremu
    Bosselut, Antoine
    Chandu, Khyathi Raghavi
    Clinciu, Miruna
    Das, Dipanjan
    Dhole, Kaustubh D.
    Du, Wanyu
    Durmus, Esin
    Gangal, Varun
    Garbacea, Cristina
    Hashimoto, Tatsunori
    Hou, Yufang
    Jernite, Yacine
    Jhamtani, Harsh
    Ji, Yangfeng
    Jolly, Shailza
    Kale, Mihir
    Kumar, Dhruv
    Ladhak, Faisal
    Madaan, Aman
    Maddela, Mounica
    Mahajan, Khyati
    Mahamood, Saad
    Majumder, Bodhisattwa Prasad
    Martins, Pedro Henrique
    McMillan-Major, Angelina
    Mille, Simon
    van Miltenburg, Emiel
    Nadeem, Moin
    Narayan, Shashi
    Nikolaev, Vitaly
    Niyongabo, Rubungo Andre
    Osei, Salomey
    Parikh, Ankur
    Perez-Beltrachini, Laura
    Rao, Niranjan Ramesh
    Raunak, Vikas
    Rodriguez, Juan Diego
    Santhanam, Sashank
    Sedoc, Joao
    Sellam, Thibault
    Shaikh, Samira
    Shimorina, Anastasia
    Sobrevilla Cabezudo, Marco Antonio
    Strobelt, Hendrik
    Subramani, Nishant
    [J]. 1ST WORKSHOP ON NATURAL LANGUAGE GENERATION, EVALUATION, AND METRICS (GEM 2021), 2021, : 96 - 120
  • [6] A Repository of Data and Evaluation Resources for Natural Language Generation
    Belz, Anja
    Gatt, Albert
    [J]. LREC 2012 - EIGHTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION, 2012, : 4027 - 4032
  • [7] Unifying Human and Statistical Evaluation for Natural Language Generation
    Hashimoto, Tatsunori B.
    Zhang, Hugh
    Liang, Percy
    [J]. 2019 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: HUMAN LANGUAGE TECHNOLOGIES (NAACL HLT 2019), VOL. 1, 2019, : 1689 - 1701
  • [8] Evaluation in Natural Language Generation: Lessons from Referring Expression Generation
    Viethen, Jette
    Dale, Robert
    [J]. TRAITEMENT AUTOMATIQUE DES LANGUES, 2007, 48 (01): : 141 - 160
  • [9] The Glass Ceiling of Automatic Evaluation in Natural Language Generation
    Colombo, Pierre
    Peyrard, Maxime
    Noiry, Nathan
    West, Robert
    Piantanida, Pablo
    [J]. 13TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING AND THE 3RD CONFERENCE OF THE ASIA-PACIFIC CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, IJCNLP-AACL 2023, 2023, : 178 - 183
  • [10] Dynamic context generation for natural language understanding: A multifaceted knowledge approach
    Chan, SWK
    Franklin, J
    [J]. IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART A-SYSTEMS AND HUMANS, 2003, 33 (01): : 23 - 41
12345