A Comparison of Transformer-Based Language Models on NLP Benchmarks

被引：1

作者：

Greco, Candida Maria ^{[1
]}

Tagarelli, Andrea ^{[1
]}

Zumpano, Ester ^{[1
]}

机构：

[1] Univ Calabria, DIMES, Arcavacata Di Rende, CS, Italy

来源：

NATURAL LANGUAGE PROCESSING AND INFORMATION SYSTEMS (NLDB 2022) | 2022年 / 13286卷

关键词：

BERTology; Deep learning; Benchmarks;

D O I：

10.1007/978-3-031-08473-7_45

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Since the advent of BERT, Transformer-based language models (TLMs) have shown outstanding effectiveness in several NLP tasks. In this paper, we aim at bringing order to the landscape of TLMs and their performance on important benchmarks for NLP. Our analysis sheds light on the advantages that some TLMs take over the others, but also unveils issues in making a complete and fair comparison in some situations.

引用

页码：490 / 501

页数：12

共 50 条

[31] Quantifying the Bias of Transformer-Based Language Models for African American English in Masked Language Modeling
Salutari, Flavia
Ramos, Jerome
Rahmani, Hossein A.
Linguaglossa, Leonardo
Lipani, Aldo
[J]. ADVANCES IN KNOWLEDGE DISCOVERY AND DATA MINING, PAKDD 2023, PT I, 2023, 13935 : 532 - 543
[32] A Comparative Analysis of Transformer-based Protein Language Models for Remote Homology Prediction
Kabir, Anowarul
Moldwin, Asher
Shehu, Amarda
[J]. 14TH ACM CONFERENCE ON BIOINFORMATICS, COMPUTATIONAL BIOLOGY, AND HEALTH INFORMATICS, BCB 2023, 2023,
[33] Transformer-based Language Models and Homomorphic Encryption: An Intersection with BERT-tiny
Rovida, Lorenzo
Leporati, Alberto
[J]. PROCEEDINGS OF THE 10TH ACM INTERNATIONAL WORKSHOP ON SECURITY AND PRIVACY ANALYTICS, IWSPA 2024, 2024, : 3 - 13
[34] An Architecture for Accelerated Large-Scale Inference of Transformer-Based Language Models
Ganiev, Amir
Chapin, Colt
de Andrade, Anderson
Liu, Chen
[J]. 2021 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, NAACL-HLT 2021, 2021, : 163 - 169
[35] Influence of Language Proficiency on the Readability of Review Text and Transformer-based Models for Determining Language Proficiency
Sazzed, Salim
[J]. COMPANION PROCEEDINGS OF THE WEB CONFERENCE 2022, WWW 2022 COMPANION, 2022, : 881 - 886
[36] Empirical Study of Tweets Topic Classification Using Transformer-Based Language Models
Mandal, Ranju
Chen, Jinyan
Becken, Susanne
Stantic, Bela
[J]. INTELLIGENT INFORMATION AND DATABASE SYSTEMS, ACIIDS 2021, 2021, 12672 : 340 - 350
[37] Classifying Drug Ratings Using User Reviews with Transformer-Based Language Models
Shiju, Akhil
He, Zhe
[J]. 2022 IEEE 10TH INTERNATIONAL CONFERENCE ON HEALTHCARE INFORMATICS (ICHI 2022), 2022, : 163 - 169
[38] Stress Test Evaluation of Transformer-based Models in Natural Language Understanding Tasks
Aspillaga, Carlos
Carvallo, Andres
Araujo, Vladimir
[J]. PROCEEDINGS OF THE 12TH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC 2020), 2020, : 1882 - 1894
[39] Bringing order into the realm of Transformer-based language models for artificial intelligence and law
Greco, Candida M.
Tagarelli, Andrea
[J]. Artificial Intelligence and Law, 2024, 32 (04) : 863 - 1010
[40] Transformers-sklearn: a toolkit for medical language understanding with transformer-based models
Feihong Yang
Xuwen Wang
Hetong Ma
Jiao Li
[J]. BMC Medical Informatics and Decision Making, 21

← 1 2 3 4 5 →