Joint intent detection and slot filling with syntactic and semantic features using multichannel CNN-BiLSTM

被引：0
|
作者：
Muhammad, Yusuf Idris [1 ]
Salim, Naomie [1 ]
Zainal, Anazida [1 ]
机构：
[1] Faculty of Computing, Universiti Teknologi Malaysia, Johor, Skudai, Malaysia
来源：
PeerJ Computer Science | 2024年 / 10卷
关键词：
Understanding spoken language is crucial for conversational agents; with intent detection and slot filling being the primary tasks in natural language understanding (NLU). Enhancing the NLU tasks can lead to an accurate and efficient virtual assistant thereby reducing the need for human intervention and expanding their applicability in other domains. Traditionally; these tasks have been addressed individually; but recent studies have highlighted their interconnection; suggesting better results when solved together. Recent advances in natural language processing have shown that pretrained word embeddings can enhance text representation and improve the generalization capabilities of models. However; the challenge of poor generalization in joint learning models for intent detection and slot filling remains due to limited annotated datasets. Additionally; traditional models face difficulties in capturing both the semantic and syntactic nuances of language; which are vital for accurate intent detection and slot filling. This study proposes a hybridized text representation method using a multichannel convolutional neural network with three embedding channels: non-contextual embeddings for semantic information; part-of-speech (POS) tag embeddings for syntactic features; and contextual embeddings for deeper contextual understanding. Specifically; we utilized word2vec for non-contextual embeddings; one-hot vectors for POS tags; and bidirectional encoder representations from transformers (BERT) for contextual embeddings. These embeddings are processed through a convolutional layer and a shared bidirectional long short-term memory (BiLSTM) network; followed by two softmax functions for intent detection and slot filling. Experiments on the air travel information system (ATIS) and SNIPS datasets demonstrated that our model significantly outperformed the baseline models; achieving an intent accuracy of 97.90% and slot filling F1-score of 98.86% on the ATIS dataset; and an intent accuracy of 98.88% and slot filling F1-score of 97.07% on the SNIPS dataset. These results highlight the effectiveness of our proposed approach in advancing dialogue systems; and paving the way for more accurate and efficient natural language understanding in real-world applications. © (2024); (PeerJ Inc.). All rights reserved;
D O I：
10.7717/PEERJ-CS.2346
中图分类号：
学科分类号：
摘要：
引用
收藏
相关论文
共 50 条
  • [41] Focus on Interaction: A Novel Dynamic Graph Model for Joint Multiple Intent Detection and Slot Filling
    Ding, Zeyuan
    Yang, Zhihao
    Lin, Hongfei
    Wang, Jian
    PROCEEDINGS OF THE THIRTIETH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE, IJCAI 2021, 2021, : 3801 - 3807
  • [42] Promoting Unified Generative Framework with Descriptive Prompts for Joint Multi-Intent Detection and Slot Filling
    Ma, Zhiyuan
    Qin, Jiwei
    Pan, Meiqi
    Tang, Song
    Mi, Jinpeng
    Liu, Dan
    ELECTRONICS, 2024, 13 (06)
  • [43] A multi-dimensional hybrid CNN-BiLSTM framework for epileptic seizure detection using electroencephalogram signal scrutiny
    Britto, K. R. Aravind
    Srinivasan, Saravanan
    Mathivanan, Sandeep Kumar
    Venkatesan, Muthukumaran
    Malar, M. B. Benjula Anbu
    Mallik, Saurav
    Qin, Hong
    SYSTEMS AND SOFT COMPUTING, 2023, 5
  • [44] LAGIM: A Label-Aware Graph Interaction Model for Joint Multiple Intent Detection and Slot Filling
    Li, Penghua
    Huang, Ziheng
    2023 35TH CHINESE CONTROL AND DECISION CONFERENCE, CCDC, 2023, : 448 - 453
  • [45] End-to-end masked graph-based CRF for joint slot filling and intent detection
    Tang, Hao
    Ji, Donghong
    Zhou, Qiji
    NEUROCOMPUTING, 2020, 413 (413) : 348 - 359
  • [46] Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Filling
    Hou, Yutai
    Lai, Yongkui
    Chen, Cheng
    Che, Wanxiang
    Liu, Ting
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL-IJCNLP 2021, 2021, : 3190 - 3200
  • [47] CBF-IDS: Addressing Class Imbalance Using CNN-BiLSTM with Focal Loss in Network Intrusion Detection System
    Peng, Haonan
    Wu, Chunming
    Xiao, Yanfeng
    APPLIED SCIENCES-BASEL, 2023, 13 (21):
  • [48] Zero Trust Network Intrusion Detection System (NIDS) using Auto Encoder for Attention-based CNN-BiLSTM
    Alalmaie, Abeer Z.
    Nanda, Priyadarsi
    He, Xiangjian
    PROCEEDINGS OF 2023 AUSTRALIAN COMPUTER SCIENCE WEEK, ACSW 2023, 2023, : 1 - 9
  • [49] Natural language understanding approaches based on joint task of intent detection and slot filling for IoT voice interaction
    Pin Ni
    Yuming Li
    Gangmin Li
    Victor Chang
    Neural Computing and Applications, 2020, 32 : 16149 - 16166
  • [50] Joint modeling method of question intent detection and slot filling for domain-oriented question answering system
    Wang, Huiyong
    Yang, Ding
    Guo, Liang
    Zhang, Xiaoming
    DATA TECHNOLOGIES AND APPLICATIONS, 2023, 57 (05) : 696 - 718
12345