Machine Translation Evaluation: Manual Versus Automatic-A Comparative Study

被引：2

作者：

Maurya, Kaushal Kumar ^{[1
]}

Ravindran, Renjith P. ^{[1
]}

Anirudh, Ch Ram ^{[1
]}

Murthy, Kavi Narayana ^{[1
]}

机构：

[1] Univ Hyderabad, Sch Comp & Informat Sci, Hyderabad, India

来源：

DATA ENGINEERING AND COMMUNICATION TECHNOLOGY, ICDECT-2K19 | 2020年 / 1079卷

关键词：

Machine translation (MT); MT evaluation; Manual metrics; Automatic metrics;

D O I：

10.1007/978-981-15-1097-7_45

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

The quality of machine translation (MT) is best judged by humans well versed in both source and target languages. However, automatic techniques are often used as these are much faster, cheaper and language independent. The goal of this paper is to check for correlation between manual and automatic evaluation, specifically in the context of Indian languages. To the extent automatic evaluation methods correlate with the manual evaluations, we can get the best of both worlds. In this paper, we perform a comparative study of automatic evaluation metrics-BLEU, NIST, METEOR, TER and WER, against the manual evaluation metric (adequacy), for English-Hindi translation. We also attempt to estimate the manual evaluation score of a given MToutput from its automatic evaluation score. The data for the study was sourced from the Workshop on Statistical Machine Translation WMT14.

引用

页码：541 / 553

页数：13

共 50 条

[1] A Comparative Study and Analysis of Evaluation Matrices in Machine Translation
Shukla, Maitry B.
Chavada, Bhoomika
PROCEEDINGS OF THE 2019 6TH INTERNATIONAL CONFERENCE ON COMPUTING FOR SUSTAINABLE GLOBAL DEVELOPMENT (INDIACOM), 2019, : 1236 - 1239
[2] A Comparative Study on Transformer Versus Sequence to Sequence in Machine Translation
Jiang, Hao
Zhao, Su
Fang, Da
Zhang, Chao
Duan, Jianjin
MODERN INDUSTRIAL IOT, BIG DATA AND SUPPLY CHAIN, IIOTBDSC 2020, 2021, 218 : 89 - 101
[3] A machine learning approach to the automatic evaluation of machine translation
Corston-Oliver, S
Gamon, M
Brockett, C
39TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, PROCEEDINGS OF THE CONFERENCE, 2001, : 140 - 147
[4] A Summary and Comparative Study of Different Metrics for Machine Translation Evaluation
Malik, Pooja
Baghel, Anurag Singh
PROCEEDINGS OF THE 8TH INTERNATIONAL CONFERENCE CONFLUENCE 2018 ON CLOUD COMPUTING, DATA SCIENCE AND ENGINEERING, 2018, : 55 - 60
[5] An Automatic Evaluation for Online Machine Translation: Holy Quran Case Study
AlSukhni, Emad
Al-Kabi, Mohammed N.
Alsmadi, Izzat M.
INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2016, 7 (06) : 118 - 123
[6] Neutralizing the Effect of Translation Shifts on Automatic Machine Translation Evaluation
Fomicheva, Marina
Bel, Nuria
da Cunha, Iria
COMPUTATIONAL LINGUISTICS AND INTELLIGENT TEXT PROCESSING (CICLING 2015), PT I, 2015, 9041 : 596 - 607
[7] An automatic evaluation of machine translation and Slavic languages
Munkova, Dasa
Munk, Michal
2014 IEEE 8TH INTERNATIONAL CONFERENCE ON APPLICATION OF INFORMATION AND COMMUNICATION TECHNOLOGIES (AICT), 2014, : 447 - 451
[8] The METEOR metric for automatic evaluation of machine translation
Lavie, Alon
Denkowski, Michael J.
MACHINE TRANSLATION, 2009, 23 (2-3) : 105 - 115
[9] BLEU: a method for automatic evaluation of machine translation
Papineni, K
Roukos, S
Ward, T
Zhu, WJ
40TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, PROCEEDINGS OF THE CONFERENCE, 2002, : 311 - 318
[10] Linguistic measures for automatic machine translation evaluation
Giménez J.
Màrquez L.
Machine Translation, 2010, 24 (3-4) : 209 - 240

← 1 2 3 4 5 →