An Eclectic Approach for Enhancing Language Models Through Rich Embedding Features

被引:0
|
作者
Aldana-Bobadilla, Edwin [1 ,2 ]
Sosa-Sosa, Victor Jesus [2 ]
Molina-Villegas, Alejandro [1 ,3 ]
Gazca-Hernandez, Karina [2 ]
Olivas, Jose Angel [4 ]
机构
[1] CONAHCYT, Mexico City 03940, Mexico
[2] Cinvestav, Unidad Tamaulipas, Ciudad Victoria 87130, Tamaulipas, Mexico
[3] Ctr Invest Ciencias Invest Geoespacial, Mexico City 14240, Mexico
[4] Univ Castilla La Mancha, Grp SMILe, Ciudad Real 13071, Spain
来源
IEEE ACCESS | 2024年 / 12卷
关键词
Task analysis; Semantics; Transformers; Neurons; Linguistics; Self-organizing feature maps; Text analysis; Self-organizing map; word embeddings; feature extraction; natural language processing;
D O I
10.1109/ACCESS.2024.3422971
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Text processing is a fundamental aspect of Natural Language Processing (NLP) and is crucial for various applications in fields such as artificial intelligence, data science, and information retrieval. It plays a core role in language models. Most text-processing approaches focus on describing and synthesizing, to a greater or lesser degree, lexical, syntactic, and semantic properties of text in the form of numerical vectors that induce a metric space, in which, it is possible to find underlying patterns and structures related to the original text. Since each approach has strengths and weaknesses, finding a single approach that perfectly extracts representative text properties for every task and application domain is hard. This paper proposes a novel approach capable of synthesizing information from heterogeneous state-of-the-art text processing approaches into a unified representation. Encouraging results demonstrate that using this representation in popular machine-learning tasks not only leads to superior performance but also offers notable advantages in memory efficiency and preservation of underlying information of the distinct sources involved in such a representation.
引用
收藏
页码:100921 / 100938
页数:18
相关论文
共 50 条
  • [31] Rich Embedding Features for One-Shot Semantic Segmentation
    Zhang, Xiaolin
    Wei, Yunchao
    Li, Zhao
    Yan, Chenggang
    Yang, Yi
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022, 33 (11) : 6484 - 6493
  • [32] Implicit Parallelism through Deep Language Embedding
    Alexandrov, Alexander
    Katsifodimos, Asterios
    Krastev, Georgi
    Markl, Volker
    SIGMOD RECORD, 2016, 45 (01) : 51 - 58
  • [33] Implicit Parallelism through Deep Language Embedding
    Alexandrov, Alexander
    Kunft, Andreas
    Katsifodimos, Asterios
    Schueler, Felix
    Thamsen, Lauritz
    Kao, Odej
    Herb, Tobias
    Markl, Volker
    SIGMOD'15: PROCEEDINGS OF THE 2015 ACM SIGMOD INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA, 2015, : 47 - 61
  • [34] Enhancing knowledge graph embedding with type-constraint features
    Wenjie Chen
    Shuang Zhao
    Xin Zhang
    Applied Intelligence, 2023, 53 : 984 - 995
  • [35] Enhancing knowledge graph embedding with type-constraint features
    Chen, Wenjie
    Zhao, Shuang
    Zhang, Xin
    APPLIED INTELLIGENCE, 2023, 53 (01) : 984 - 995
  • [36] Intent Features for Rich Natural Language Understanding
    Lester, Brian
    Choudhury, Sagnik Ray
    Prasad, Rashmi
    Bangalore, Srinivas
    2021 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, NAACL-HLT 2021, 2021, : 214 - 221
  • [37] CALLM: Enhancing Clinical Interview Analysis Through Data Augmentation With Large Language Models
    Wu, Yuqi
    Mao, Kaining
    Zhang, Yanbo
    Chen, Jie
    IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, 2024, 28 (12) : 7531 - 7542
  • [38] Visualizing Dialogues: Enhancing Image Selection through Dialogue Understanding with Large Language Models
    Kao, Chang-Sheng
    Chen, Yun-Nung
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: ACL 2024, 2024, : 11777 - 11788
  • [39] DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment
    Lu, Ke-Han
    Chen, Zhehuai
    Fu, Szu-Wei
    Huang, He
    Ginsburg, Boris
    Wang, Yu-Chiang Frank
    Lee, Hung-yi
    INTERSPEECH 2024, 2024, : 4159 - 4163
  • [40] "Conversing" With Qualitative Data: Enhancing Qualitative Research Through Large Language Models (LLMs)
    Hayes, Adam S.
    INTERNATIONAL JOURNAL OF QUALITATIVE METHODS, 2025, 24