Learning Directional Sentence-Pair Embedding for Natural Language Reasoning (Student Abstract)

被引:0
|
作者
Jiang, Yuchen [1 ,2 ]
Xiao, Zhenxin [1 ,2 ]
Chang, Kai-Wei [1 ]
机构
[1] Univ Calif Los Angeles, Los Angeles, CA 90095 USA
[2] Zhejiang Univ, Hangzhou, Zhejiang, Peoples R China
关键词
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Enabling the models with the ability of reasoning and inference over text is one of the core missions of natural language understanding. Despite deep learning models have shown strong performance on various cross-sentence inference benchmarks, recent work has shown that they are leveraging spurious statistical cues rather than capturing deeper implied relations between pairs of sentences. In this paper, we show that the state-of-the-art language encoding models are especially bad at modeling directional relations between sentences by proposing a new evaluation task: Cause-and-Effect relation prediction task. Back by our curated Cause-and-Effect Relation dataset (CER), we also demonstrate that a mutual attention mechanism can guide the model to focus on capturing directional relations between sentences when added to existing transformer-based models. Experiment results show that the proposed approach improves the performance on downstream applications, such as the abductive reasoning task.
引用
收藏
页码:13825 / 13826
页数:2
相关论文
共 26 条
  • [1] Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding
    Barkan, Oren
    Razin, Noam
    Malkiel, Itzik
    Katz, Ori
    Caciularu, Avi
    Koenigstein, Noam
    [J]. THIRTY-FOURTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THE THIRTY-SECOND INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE AND THE TENTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2020, 34 : 3235 - 3242
  • [2] Evaluation of Sentence Embedding Models for Natural Language Understanding Problems in Russian
    Popov, Dmitry
    Pugachev, Alexander
    Svyatokum, Polina
    Svitanko, Elizaveta
    Artemova, Ekaterina
    [J]. ANALYSIS OF IMAGES, SOCIAL NETWORKS AND TEXTS, AIST 2019, 2019, 11832 : 205 - 217
  • [3] Analysis of sentence embedding models using prediction tasks in natural language processing
    Adi, Y.
    Kermany, E.
    Belinkov, Y.
    Lavi, O.
    Goldberg, Y.
    [J]. IBM JOURNAL OF RESEARCH AND DEVELOPMENT, 2017, 61 (4-5)
  • [5] lilGym: Natural Language Visual Reasoning with Reinforcement Learning
    Wu, Anne
    Brantley, Kiante
    Kojima, Noriyuki
    Artzi, Yoav
    [J]. PROCEEDINGS OF THE 61ST ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2023): LONG PAPERS, VOL 1, 2023, : 9214 - 9234
  • [6] Guiding Reinforcement Learning Exploration Using Natural Language Extended Abstract
    Harrison, Brent
    Ehsan, Upol
    Riedl, Mark O.
    [J]. PROCEEDINGS OF THE 17TH INTERNATIONAL CONFERENCE ON AUTONOMOUS AGENTS AND MULTIAGENT SYSTEMS (AAMAS' 18), 2018, : 1956 - 1958
  • [7] Automated Natural Language Explanation of Deep Visual Neurons with Large Models (Student Abstract)
    Zhao, Chenxu
    Qian, Wei
    Shi, Yucheng
    Huai, Mengdi
    Liu, Ninghao
    [J]. THIRTY-EIGTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 21, 2024, : 23712 - 23713
  • [8] Causal Knowledge Extraction from Text using Natural Language Inference (Student Abstract)
    Bhandari, Manik
    Feblowitz, Mark
    Hassanzadeh, Oktie
    Srinivas, Kavitha
    Sohrabi, Shirin
    [J]. THIRTY-FIFTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THIRTY-THIRD CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE AND THE ELEVENTH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2021, 35 : 15759 - 15760
  • [9] CONSTRUCTION OF NATURAL-LANGUAGE SENTENCE ACCEPTORS BY A SUPERVISED-LEARNING TECHNIQUE
    COULON, D
    KAYSER, D
    [J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 1979, 1 (01) : 94 - 99
  • [10] The Language Model Can Have the Personality: Joint Learning for Personality Enhanced Language Model (Student Abstract)
    Chen, Tianyi
    Cao, Feiqi
    Ding, Yihao
    Han, Caren
    [J]. THIRTY-EIGTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 21, 2024, : 23454 - 23455