A Comprehensive Survey on Various Fully Automatic Machine Translation Evaluation Metrics

被引:13
|
作者
Chauhan, Shweta [1 ]
Daniel, Philemon [1 ]
机构
[1] Natl Inst Technol, Dept Elect & Commun, Hamirpur 177005, Himachal Prades, India
关键词
Machine translation evaluation; Machine translation; Automated metrics; Metrics; SYSTEMS;
D O I
10.1007/s11063-022-10835-4
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The fast advancement in machine translation models necessitates the development of accurate evaluation metrics that would allow researchers to track the progress in text languages. The evaluation of machine translation models is crucial since its results are exploited for improvements of translation models. However fully automatically evaluating the machine translation models in itself is a huge challenge for the researchers as human evaluation is very expensive, time-consuming, unreproducible. This paper presents a detailed classification and comprehensive survey on various fully automated evaluation metrics, which are used to assess the performance or quality of machine translated output. Various fully automatic evaluation metrics are classified into five categories that are lexical, character, semantic, syntactic, and semantic & syntactic evaluation metrics for better understanding purpose. Taking account of the challenges posed in the field of machine translation evaluation by Statistical Machine Translation and Neural Machine Translation, along with a discussion on the advantages, disadvantages, and gaps for each fully automatic machine translation evaluation metric has been provided. The presented study will help machine translation researchers in quickly identifying automatic machine translation evaluation metrics that are most appropriate for the improvement or development of their machine translation model, as well as researchers in gaining a general understanding of how automatic machine translation evaluation research evolved.
引用
收藏
页码:12663 / 12717
页数:55
相关论文
共 50 条
  • [31] Optimizing Automatic Evaluation of Machine Translation with the ListMLE Approach
    Li, Maoxi
    Wang, Mingwen
    ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING, 2019, 18 (01)
  • [32] Automatic Evaluation and Analysis of Idioms in Neural Machine Translation
    Baziotis, Christos
    Mathur, Prashant
    Hasler, Eva
    17TH CONFERENCE OF THE EUROPEAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, EACL 2023, 2023, : 3682 - 3700
  • [33] VERTa: a linguistic approach to automatic machine translation evaluation
    Comelles, Elisabet
    Atserias, Jordi
    LANGUAGE RESOURCES AND EVALUATION, 2019, 53 (01) : 57 - 86
  • [34] Automatic Evaluation of Machine Translation Output for Slovak Language
    Kasas, Karol
    Munkova, Dasa
    DIVAI 2016: 11TH INTERNATIONAL SCIENTIFIC CONFERENCE ON DISTANCE LEARNING IN APPLIED INFORMATICS, 2016, : 533 - 540
  • [35] Automatic Evaluation of Machine Translation Through the Residual Analysis
    Munkova, Dasa
    Munk, Michal
    ADVANCED INTELLIGENT COMPUTING THEORIES AND APPLICATIONS, ICIC 2015, PT III, 2015, 9227 : 481 - 490
  • [36] A comprehensive survey on feature selection in the various fields of machine learning
    Pradip Dhal
    Chandrashekhar Azad
    Applied Intelligence, 2022, 52 : 4543 - 4581
  • [37] A comprehensive survey on feature selection in the various fields of machine learning
    Dhal, Pradip
    Azad, Chandrashekhar
    APPLIED INTELLIGENCE, 2022, 52 (04) : 4543 - 4581
  • [38] Guardians of the Machine Translation Meta-Evaluation: Sentinel Metrics Fall In!
    Perrellai, Stefano
    Proietti, Lorenzo
    Scire, Alessandro
    Barba, Edoardo
    Navigli, Roberto
    PROCEEDINGS OF THE 62ND ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 1: LONG PAPERS, 2024, : 16216 - 16244
  • [39] A review of machine transliteration, translation, evaluation metrics and datasets in Indian Languages
    Jha, Abhinav
    Patil, Hemprasad Yashwant
    MULTIMEDIA TOOLS AND APPLICATIONS, 2023, 82 (15) : 23509 - 23540
  • [40] Optimizing Non-Decomposable Evaluation Metrics for Neural Machine Translation
    Shen, Shi-Qi
    Liu, Yang
    Sun, Mao-Song
    JOURNAL OF COMPUTER SCIENCE AND TECHNOLOGY, 2017, 32 (04) : 796 - 804