Adversarial Training for Unknown Word Problems in Neural Machine Translation

被引:4
|
作者
Ji, Yatu [1 ]
Hou, Hongxu [1 ]
Chen, Junjie [1 ]
Wu, Nier [1 ]
机构
[1] Inner Mongolia Univ, Comp Sci Dept, Hohhot 010021, Peoples R China
关键词
Neural machine translation; UNK; generative adversarial network; value iteration;
D O I
10.1145/3342482
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Nearly all of the work in neural machine translation (NMT) is limited to a quite restricted vocabulary, crudely treating all other words the same as an < unk > symbol. For the translation of language with abundant morphology, unknown (UNK) words also come from the misunderstanding of the translation model to the morphological changes. In this study, we explore two ways to alleviate the UNK problem in NMT: a new generative adversarial network (added value constraints and semantic enhancement) and a preprocessing technique that mixes morphological noise. The training process is like a win-win game in which the players are three adversarial sub models (generator, filter, and discriminator). In this game, the filter is to emphasize the discriminator's attention to the negative generations that contain noise and improve the training efficiency. Finally, the discriminator cannot easily discriminate the negative samples generated by the generator with filter and human translations. The experimental results show that the proposed method significantly improves over several strong baseline models across various language pairs and the newly emerged Mongolian-Chinese task is state-of-the-art.
引用
收藏
页数:12
相关论文
共 50 条
  • [1] Generative adversarial training for neural machine translation
    Yang, Zhen
    Chen, Wei
    Wang, Feng
    Xu, Bo
    [J]. NEUROCOMPUTING, 2018, 321 : 146 - 155
  • [2] Noise-Based Adversarial Training for Enhancing Agglutinative Neural Machine Translation
    Ji, Yatu
    Hou, Hongxu
    Chen, Junjie
    Wu, Nier
    [J]. PRICAI 2019: TRENDS IN ARTIFICIAL INTELLIGENCE, PT I, 2019, 11670 : 392 - 396
  • [3] Crafting Adversarial Examples for Neural Machine Translation
    Zhang, Xinze
    Zhang, Junzhe
    Chen, Zhenhua
    He, Kun
    [J]. 59TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS AND THE 11TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING, VOL 1 (ACL-IJCNLP 2021), 2021, : 1967 - 1977
  • [4] Manifold Adversarial Augmentation for Neural Machine Translation
    Chen, Guandan
    Fan, Kai
    Zhang, Kaibo
    Chen, Boxing
    Huang, Zhongqiang
    [J]. FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL-IJCNLP 2021, 2021, : 3184 - 3189
  • [5] Effective Adversarial Regularization for Neural Machine Translation
    Sato, Motoki
    Suzuki, Jun
    Kiyono, Shun
    [J]. 57TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2019), 2019, : 204 - 210
  • [6] Incorporating Statistical Machine Translation Word Knowledge Into Neural Machine Translation
    Wang, Xing
    Tu, Zhaopeng
    Zhang, Min
    [J]. IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2018, 26 (12) : 2255 - 2266
  • [7] A4NT: Author Attribute Anonymity by Adversarial Training of Neural Machine Translation
    Shetty, Rakshith
    Schiele, Bernt
    Fritz, Mario
    [J]. PROCEEDINGS OF THE 27TH USENIX SECURITY SYMPOSIUM, 2018, : 1633 - 1650
  • [8] On the Word Alignment from Neural Machine Translation
    Li, Xintong
    Li, Guanlin
    Liu, Lemao
    Meng, Max
    Shi, Shuming
    [J]. 57TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2019), 2019, : 1293 - 1303
  • [9] Content Word Aware Neural Machine Translation
    Chen, Kehai
    Wang, Rui
    Utiyama, Masao
    Sumita, Eiichiro
    [J]. 58TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2020), 2020, : 358 - 364
  • [10] Word Position Aware Translation Memory for Neural Machine Translation
    He, Qiuxiang
    Huang, Guoping
    Liu, Lemao
    Li, Li
    [J]. NATURAL LANGUAGE PROCESSING AND CHINESE COMPUTING (NLPCC 2019), PT I, 2019, 11838 : 367 - 379