mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences

被引：0

作者：

Uthus, David ^{[1
]}

Ontanion, Santiago ^{[1
]}

Ainslie, Joshua ^{[1
]}

Guo, Mandy ^{[1
]}

机构：

[1] Google Res, Mountain View, CA 94043 USA

来源：

FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (EMNLP 2023) | 2023年

关键词：

D O I：

暂无

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We present our work on developing a multilingual, efficient text-to-text transformer that is suitable for handling long inputs. This model, called mLongT5, builds upon the architecture of LongT5, while leveraging the multilingual datasets used for pretraining mT5 and the pretraining tasks of UL2. We evaluate this model on a variety of multilingual summarization and question-answering tasks, and the results show stronger performance for mLongT5 when compared to existing multilingual models such as mBART or M-BERT.

引用

页码：9380 / 9386

页数：7

共 50 条

[31] LegalT5-ABSA: a framework for aspect-based sentiment analysis of parties in legal cases using text-to-text transfer transformer
Melal, Sevda Rezaei
Melal, Sepehr Rezaei
Khanjani-Shiraz, Rashed
INTERNATIONAL JOURNAL OF DATA SCIENCE AND ANALYTICS, 2025,
[32] NegT5: A Cross-Task Text-to-Text Framework for Negation in Question Answering
Jin, Tao
Racharak, Teeradaj
Minh Le Nguyen
INTELLIGENT INFORMATION AND DATABASE SYSTEMS, ACIIDS 2023, PT II, 2023, 13996 : 272 - 285
[33] Employing a Multilingual Transformer Model for Segmenting Unpunctuated Arabic Text
Alshanqiti, Abdullah M.
Albouq, Sami
Alkhodre, Ahmad B.
Namoun, Abdallah
Nabil, Emad
APPLIED SCIENCES-BASEL, 2022, 12 (20):
[34] Multilingual Text Summarization for German Texts Using Transformer Models
Alcantara, Tomas Humberto Montiel
Krutli, David
Ravada, Revathi
Hanne, Thomas
INFORMATION, 2023, 14 (06)
[35] Emotion recognition in Hindi text using multilingual BERT transformer
Kumar, Tapesh
Mahrishi, Mehul
Sharma, Girish
MULTIMEDIA TOOLS AND APPLICATIONS, 2023, 82 (27) : 42373 - 42394
[36] Emotion recognition in Hindi text using multilingual BERT transformer
Tapesh Kumar
Mehul Mahrishi
Girish Sharma
Multimedia Tools and Applications, 2023, 82 : 42373 - 42394
[37] New Encoding Schemes for Efficient Multilingual Text Messaging
Jalan, Ankit
Rajawat, Ketan
Hegde, Rajesh M.
2014 TWENTIETH NATIONAL CONFERENCE ON COMMUNICATIONS (NCC), 2014,
[38] Sentence-T5 (ST5): Scalable Sentence Encoders from Pre-trained Text-to-Text Models
Ni, Jianmo
Abrego, Gustavo Hernandez
Constant, Noah
Ma, Ji
Hall, Keith B.
Cer, Daniel
Yang, Yinfei
FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2022), 2022, : 1864 - 1874
[39] Rethinking Efficient Multilingual Text Summarization Meta-Evaluation
Han, Rilyn R.
Chen, Jiawen
Liu, Yixin
Cohan, Arman
FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: ACL 2024, 2024, : 15739 - 15746
[40] EXTRACTING SEVERITY MARKERS FROM UNSTRUCTURED CLINICAL DATA OF CONGESTIVE HEART FAILURE PATIENTS USING A PRETRAINED TEXT-TO-TEXT TRANSFER TRANSFORMER MODEL
Kumar, V
Rasouliyan, L.
Althoff, A. G.
Long, S.
Zema, C.
Rao, M. B.
VALUE IN HEALTH, 2022, 25 (07) : S526 - S526

← 1 2 3 4 5 →