Sparse and Constrained Attention for Neural Machine Translation

被引：0

作者：

Malaviya, Chaitanya ^{[1
,3
]}

Ferreira, Pedro ^{[2
]}

Martins, Andre F. T. ^{[3
,4
]}

机构：

[1] Carnegie Mellon Univ, Language Technol Inst, Pittsburgh, PA 15213 USA

[2] Univ Lisbon, Inst Super Tecn, Lisbon, Portugal

[3] Unbabel, Lisbon, Portugal

[4] Inst Telecomunicacoes, Lisbon, Portugal

来源：

PROCEEDINGS OF THE 56TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 2 | 2018年

基金：

欧洲研究理事会;

关键词：

D O I：

暂无

中图分类号：

TP39 [计算机的应用];

学科分类号：

081203 ; 0835 ;

摘要：

In NMT, words are sometimes dropped from the source or generated repeatedly in the translation. We explore novel strategies to address the coverage problem that change only the attention transformation. Our approach allocates fertilities to source words, used to bound the attention each word can receive. We experiment with various sparse and constrained attention transformations and propose a new one, constrained sparsemax, shown to be differentiable and sparse. Empirical evaluation is provided in three languages pairs.

引用

页码：370 / 376

页数：7

共 50 条

[1] Neural Machine Translation with Deep Attention
Zhang, Biao
Xiong, Deyi
Su, Jinsong
[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2020, 42 (01) : 154 - 163
[2] Recurrent Attention for Neural Machine Translation
Zeng, Jiali
Wu, Shuangzhi
Yin, Yongjing
Jiang, Yufan
Li, Mu
[J]. 2021 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2021), 2021, : 3216 - 3225
[3] Attention-via-Attention Neural Machine Translation
Zhao, Shenjian
Zhang, Zhihua
[J]. THIRTY-SECOND AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTIETH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / EIGHTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2018, : 563 - 570
[4] Bilingual attention based neural machine translation
Kang, Liyan
He, Shaojie
Wang, Mingxuan
Long, Fei
Su, Jinsong
[J]. APPLIED INTELLIGENCE, 2023, 53 (04) : 4302 - 4315
[5] Bilingual attention based neural machine translation
Liyan Kang
Shaojie He
Mingxuan Wang
Fei Long
Jinsong Su
[J]. Applied Intelligence, 2023, 53 : 4302 - 4315
[6] Attention Calibration for Transformer in Neural Machine Translation
Lu, Yu
Zeng, Jiali
Zhang, Jiajun
Wu, Shuangzhi
Li, Mu
[J]. 59TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS AND THE 11TH INTERNATIONAL JOINT CONFERENCE ON NATURAL LANGUAGE PROCESSING, VOL 1 (ACL-IJCNLP 2021), 2021, : 1288 - 1298
[7] Parallel Attention Mechanisms in Neural Machine Translation
Medina, Julian Richard
Kalita, Jugal
[J]. 2018 17TH IEEE INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND APPLICATIONS (ICMLA), 2018, : 547 - 552
[8] Lexically Constrained Neural Machine Translation with Levenshtein Transformer
Susanto, Raymond Hendy
Chollampatt, Shamil
Tan, Liling
[J]. 58TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2020), 2020, : 3536 - 3543
[9] Syntax-Directed Attention for Neural Machine Translation
Chen, Kehai
Wang, Rui
Utiyama, Masao
Sumita, Eiichiro
Zhao, Tiejun
[J]. THIRTY-SECOND AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTIETH INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / EIGHTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2018, : 4792 - 4799
[10] Neural Machine Translation with Target-Attention Model
Yang, Mingming
Zhang, Min
Chen, Kehai
Wang, Rui
Zhao, Tiejun
[J]. IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2020, E103D (03) : 684 - 694

← 1 2 3 4 5 →