Visual Exploration of Semantic Relationships in Neural Word Embeddings

被引:61
|
作者
Liu, Shusen [1 ]
Bremer, Peer-Timo [1 ]
Thiagarajan, Jayaraman J. [1 ]
Srikumar, Vivek [3 ]
Wang, Bei [2 ]
Livnat, Yarden [2 ]
Pascucci, Valerio [2 ]
机构
[1] Lawrence Livermore Natl Lab, Lawrence, CA 94550 USA
[2] Univ Utah, SCI Inst, Salt Lake City, UT 84112 USA
[3] Univ Utah, Sch Comp, Salt Lake City, UT 84112 USA
基金
美国国家科学基金会;
关键词
Natural Language Processing; Word Embedding; High-Dimensional Data; DIMENSIONALITY REDUCTION; VISUALIZATION; QUALITY;
D O I
10.1109/TVCG.2017.2745141
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Constructing distributed representations for words through neural language models and using the resulting vector spaces for analysis has become a crucial component of natural language processing (NLP). However, despite their widespread application, little is known about the structure and properties of these spaces. To gain insights into the relationship between words, the NLP community has begun to adapt high-dimensional visualization techniques. In particular, researchers commonly use t-distributed stochastic neighbor embeddings (t-SNE) and principal component analysis (PCA) to create two-dimensional embeddings for assessing the overall structure and exploring linear relationships (e.g., word analogies), respectively. Unfortunately, these techniques often produce mediocre or even misleading results and cannot address domain-specific visualization challenges that are crucial for understanding semantic relationships in word embeddings. Here, we introduce new embedding techniques for visualizing semantic and syntactic analogies, and the corresponding tests to determine whether the resulting views capture salient structures. Additionally, we introduce two novel views for a comprehensive study of analogy relationships. Finally, we augment t-SNE embeddings to convey uncertainty information in order to allow a reliable interpretation. Combined, the different views address a number of domain-specific tasks difficult to solve with existing tools.
引用
收藏
页码:553 / 562
页数:10
相关论文
共 50 条
  • [1] An Exploration of Semantic Relations in Neural Word Embeddings Using Extrinsic Knowledge
    Chen, Zhiwei
    He, Zhe
    Liu, Xiuwen
    Bian, Jiang
    [J]. 2017 IEEE INTERNATIONAL CONFERENCE ON BIOINFORMATICS AND BIOMEDICINE (BIBM), 2017, : 1246 - 1251
  • [2] Visual exploration and comparison of word embeddings
    Chen, Juntian
    Tao, Yubo
    Lin, Hai
    [J]. JOURNAL OF VISUAL LANGUAGES AND COMPUTING, 2018, 48 : 178 - 186
  • [3] Visual Exploration of Relationships and Structure in Low-Dimensional Embeddings
    Eckelt, Klaus
    Hinterreiter, Andreas
    Adelberger, Patrick
    Walchshofer, Conny
    Dhanoa, Vaishali
    Humer, Christina
    Heckmann, Moritz
    Steinparz, Christian
    Streit, Marc
    [J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 2023, 29 (07) : 3312 - 3326
  • [4] Semantic Structure and Interpretability of Word Embeddings
    Senel, Lutfi Kerem
    Utlu, Ihsan
    Yucesoy, Veysel
    Koc, Aykut
    Cukur, Tolga
    [J]. IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING, 2018, 26 (10) : 1769 - 1779
  • [5] Semantic Word Cloud Generation Based on Word Embeddings
    Xu, Jin
    Tao, Yubo
    Lin, Hai
    [J]. 2016 IEEE PACIFIC VISUALIZATION SYMPOSIUM (PACIFICVIS), 2016, : 239 - 243
  • [6] Evaluating semantic relations in neural word embeddings with biomedical and general domain knowledge bases
    Zhiwei Chen
    Zhe He
    Xiuwen Liu
    Jiang Bian
    [J]. BMC Medical Informatics and Decision Making, 18
  • [7] Improved Learning of Word Embeddings with Word Definitions and Semantic Injection
    Zhang, Yichi
    Dai, Yinpei
    Ou, Zhijian
    Wang, Huixin
    Feng, Junlan
    [J]. INTERSPEECH 2020, 2020, : 4253 - 4257
  • [8] Evaluating semantic relations in neural word embeddings with biomedical and general domain knowledge bases
    Chen, Zhiwei
    He, Zhe
    Liu, Xiuwen
    Bian, Jiang
    [J]. BMC MEDICAL INFORMATICS AND DECISION MAKING, 2018, 18
  • [9] Learning Semantic Hierarchies via Word Embeddings
    Fu, Ruiji
    Guo, Jiang
    Qin, Bing
    Che, Wanxiang
    Wang, Haifeng
    Liu, Ting
    [J]. PROCEEDINGS OF THE 52ND ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 1, 2014, : 1199 - 1209
  • [10] Dynamic Word Embeddings for Evolving Semantic Discovery
    Yao, Zijun
    Sun, Yifan
    Ding, Weicong
    Rao, Nikhil
    Xiong, Hui
    [J]. WSDM'18: PROCEEDINGS OF THE ELEVENTH ACM INTERNATIONAL CONFERENCE ON WEB SEARCH AND DATA MINING, 2018, : 673 - 681