Joint intent detection and slot filling with syntactic and semantic features using multichannel CNN-BiLSTM

被引：0
|
作者：
Muhammad, Yusuf Idris [1 ]
Salim, Naomie [1 ]
Zainal, Anazida [1 ]
机构：
[1] Faculty of Computing, Universiti Teknologi Malaysia, Johor, Skudai, Malaysia
来源：
PeerJ Computer Science | 2024年 / 10卷
关键词：
Understanding spoken language is crucial for conversational agents; with intent detection and slot filling being the primary tasks in natural language understanding (NLU). Enhancing the NLU tasks can lead to an accurate and efficient virtual assistant thereby reducing the need for human intervention and expanding their applicability in other domains. Traditionally; these tasks have been addressed individually; but recent studies have highlighted their interconnection; suggesting better results when solved together. Recent advances in natural language processing have shown that pretrained word embeddings can enhance text representation and improve the generalization capabilities of models. However; the challenge of poor generalization in joint learning models for intent detection and slot filling remains due to limited annotated datasets. Additionally; traditional models face difficulties in capturing both the semantic and syntactic nuances of language; which are vital for accurate intent detection and slot filling. This study proposes a hybridized text representation method using a multichannel convolutional neural network with three embedding channels: non-contextual embeddings for semantic information; part-of-speech (POS) tag embeddings for syntactic features; and contextual embeddings for deeper contextual understanding. Specifically; we utilized word2vec for non-contextual embeddings; one-hot vectors for POS tags; and bidirectional encoder representations from transformers (BERT) for contextual embeddings. These embeddings are processed through a convolutional layer and a shared bidirectional long short-term memory (BiLSTM) network; followed by two softmax functions for intent detection and slot filling. Experiments on the air travel information system (ATIS) and SNIPS datasets demonstrated that our model significantly outperformed the baseline models; achieving an intent accuracy of 97.90% and slot filling F1-score of 98.86% on the ATIS dataset; and an intent accuracy of 98.88% and slot filling F1-score of 97.07% on the SNIPS dataset. These results highlight the effectiveness of our proposed approach in advancing dialogue systems; and paving the way for more accurate and efficient natural language understanding in real-world applications. © (2024); (PeerJ Inc.). All rights reserved;
D O I：
10.7717/PEERJ-CS.2346
中图分类号：
学科分类号：
摘要：
引用
收藏
相关论文
共 50 条
  • [21] Joint intent detection and slot filling with wheel-graph attention networks
    Wei, Pengfei
    Zeng, Bi
    Liao, Wenxiong
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2022, 42 (03) : 2409 - 2420
  • [22] A Fast Attention Network for Joint Intent Detection and Slot Filling on Edge Devices
    Huang L.
    Liang S.
    Ye F.
    Gao N.
    IEEE Transactions on Artificial Intelligence, 2024, 5 (02): : 530 - 540
  • [23] ACJIS: A Novel Attentive Cross Approach For Joint Intent Detection And Slot Filling
    Yu, Shuai
    Shen, Lei
    Zhu, Pengcheng
    Chen, Jiansong
    2018 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN), 2018,
  • [24] A Survey of Joint Intent Detection and Slot Filling Models in Natural Language Understanding
    Weld, Henry
    Huang, Xiaoqi
    Long, Siqu
    Poon, Josiah
    Han, Soyeon Caren
    ACM COMPUTING SURVEYS, 2023, 55 (08)
  • [25] Deep CNN-BiLSTM Model for Transportation Mode Detection Using Smartphone Accelerometer and Magnetometer
    Tang, Qinrui
    Jahan, Kanwal
    Roth, Michael
    2022 IEEE INTELLIGENT VEHICLES SYMPOSIUM (IV), 2022, : 772 - 778
  • [26] Pre-Trained Joint Model for Intent Classification and Slot Filling with Semantic Feature Fusion
    Chen, Yan
    Luo, Zhenghang
    SENSORS, 2023, 23 (05)
  • [27] Joint Intention Detection and Semantic Slot Filling Based on BLSTM and Attention
    Chen Tingting
    Lin Min
    Li Yanling
    2019 IEEE 4TH INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND BIG DATA ANALYSIS (ICCCBDA), 2019, : 690 - 694
  • [28] A Two-Stage Selective Fusion Framework for Joint Intent Detection and Slot Filling
    Ma, Ziyu
    Sun, Bin
    Li, Shutao
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2024, 35 (03) : 3874 - 3885
  • [29] SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling
    Wu, Di
    Ding, Liang
    Lu, Fan
    Xie, Jian
    PROCEEDINGS OF THE 2020 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP), 2020, : 1932 - 1937
  • [30] Joint Training Model of Intent Detection and Slot Filling for Multi Granularity Implicit Guidance
    Li, Bin
    Wang, Weihua
    Bao, Feilong
    2022 INTERNATIONAL CONFERENCE ON ASIAN LANGUAGE PROCESSING (IALP 2022), 2022, : 271 - 274
12345