A Repository of Data and Evaluation Resources for Natural Language Generation

被引:0
|
作者
Belz, Anja [1 ]
Gatt, Albert [2 ]
机构
[1] Univ Brighton, Lewes Rd, Brighton BN2 4GJ, E Sussex, England
[2] Univ Malta, MSD-2080 Msida, Malta
关键词
Natural Language Generation; Evaluation Resources; Data Resources; REFERRING EXPRESSIONS;
D O I
暂无
中图分类号
H0 [语言学];
学科分类号
030303 ; 0501 ; 050102 ;
摘要
Starting in 2007, the field of natural language generation (NLG) has organised shared-task evaluation events every year, under the Generation Challenges umbrella. In the course of these shared tasks, a wealth of data has been created, along with associated task definitions and evaluation regimes. In other contexts too, sharable NLG data is now being created. In this paper, we describe the online repository that we have created as a one-stop resource for obtaining NLG task materials, both from Generation Challenges tasks and from other sources, where the set of materials provided for each task consists of (i) task definition, (ii) input and output data, (iii) evaluation software, (iv) documentation, and (v) publications reporting previous results.
引用
收藏
页码:4027 / 4032
页数:6
相关论文
共 50 条
  • [1] A Data Category Repository for Language Resources
    Warburton, Kara
    Wright, Sue Ellen
    [J]. DEVELOPMENT OF LINGUISTIC LINKED OPEN DATA RESOURCES FOR COLLABORATIVE DATA-INTENSIVE RESEARCH IN THE LANGUAGE SCIENCES, 2019, : 69 - 97
  • [2] Evaluation in the context of natural language generation
    Mellish, C
    Dale, R
    [J]. COMPUTER SPEECH AND LANGUAGE, 1998, 12 (04): : 349 - 373
  • [3] IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation
    Cahyawijaya, Samuel
    Winata, Genta Indra
    Wilie, Bryan
    Vincentio, Karissa
    Li, Xiaohong
    Kuncoro, Adhiguna
    Ruder, Sebastian
    Lim, Zhi Yuan
    Bahar, Syafri
    Khodra, Masayu Leylia
    Purwarianti, Ayu
    Fung, Pascale
    [J]. 2021 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2021), 2021, : 8875 - 8898
  • [4] Natural Language Generation Using Automatically Constructed Lexical Resources
    Ito, Naho
    Hagiwara, Masafumi
    [J]. 2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2011, : 980 - 987
  • [5] An Overview of Natural Language Generation Systems Evaluation
    Yang, Feng-Jen
    [J]. WORLD CONGRESS ON ENGINEERING AND COMPUTER SCIENCE, WCECS 2015, VOL I, 2015, : 71 - 74
  • [6] Evaluating the Evaluation of Diversity in Natural Language Generation
    Tevet, Guy
    Berant, Jonathan
    [J]. 16TH CONFERENCE OF THE EUROPEAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EACL 2021), 2021, : 326 - 346
  • [7] Natural Language Generation, its Evaluation and Metrics
    Gehrmann, Sebastian
    Adewumi, Tosin
    Aggarwal, Karmanya
    Ammanamanchi, Pawan Sasanka
    Anuoluwapo, Aremu
    Bosselut, Antoine
    Chandu, Khyathi Raghavi
    Clinciu, Miruna
    Das, Dipanjan
    Dhole, Kaustubh D.
    Du, Wanyu
    Durmus, Esin
    Gangal, Varun
    Garbacea, Cristina
    Hashimoto, Tatsunori
    Hou, Yufang
    Jernite, Yacine
    Jhamtani, Harsh
    Ji, Yangfeng
    Jolly, Shailza
    Kale, Mihir
    Kumar, Dhruv
    Ladhak, Faisal
    Madaan, Aman
    Maddela, Mounica
    Mahajan, Khyati
    Mahamood, Saad
    Majumder, Bodhisattwa Prasad
    Martins, Pedro Henrique
    McMillan-Major, Angelina
    Mille, Simon
    van Miltenburg, Emiel
    Nadeem, Moin
    Narayan, Shashi
    Nikolaev, Vitaly
    Niyongabo, Rubungo Andre
    Osei, Salomey
    Parikh, Ankur
    Perez-Beltrachini, Laura
    Rao, Niranjan Ramesh
    Raunak, Vikas
    Rodriguez, Juan Diego
    Santhanam, Sashank
    Sedoc, Joao
    Sellam, Thibault
    Shaikh, Samira
    Shimorina, Anastasia
    Sobrevilla Cabezudo, Marco Antonio
    Strobelt, Hendrik
    Subramani, Nishant
    [J]. 1ST WORKSHOP ON NATURAL LANGUAGE GENERATION, EVALUATION, AND METRICS (GEM 2021), 2021, : 96 - 120
  • [8] Integration of data based on ontologies for the generation of a multimedia repository of language and speech.
    Quisi-Peralta, Diego
    Galan-Mena, Jorge
    Lopez-Nores, Martin
    Chaglla-Rodriguez, Liliana
    Robles-Bykbaev, Vladimir
    [J]. 2018 CONGRESO INTERNACIONAL DE INNOVACION Y TENDENCIAS EN INGENIERIA (CONIITI), 2018,
  • [9] Market Data Resources for Researchers: The SIRCA Data Repository
    Peat, Maurice
    [J]. AUSTRALIAN ECONOMIC REVIEW, 2009, 42 (04) : 490 - 495
  • [10] Towards a comprehensive open repository of Polish language resources
    Ogrodniczuk, Maciej
    Pezik, Piotr
    Przepiorkowski, Adam
    [J]. LREC 2012 - EIGHTH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION, 2012, : 3593 - 3597