Joint intent detection and slot filling with syntactic and semantic features using multichannel CNN-BiLSTM

被引：0
|
作者：
Muhammad, Yusuf Idris [1 ]
Salim, Naomie [1 ]
Zainal, Anazida [1 ]
机构：
[1] Faculty of Computing, Universiti Teknologi Malaysia, Johor, Skudai, Malaysia
来源：
PeerJ Computer Science | 2024年 / 10卷
关键词：
Understanding spoken language is crucial for conversational agents; with intent detection and slot filling being the primary tasks in natural language understanding (NLU). Enhancing the NLU tasks can lead to an accurate and efficient virtual assistant thereby reducing the need for human intervention and expanding their applicability in other domains. Traditionally; these tasks have been addressed individually; but recent studies have highlighted their interconnection; suggesting better results when solved together. Recent advances in natural language processing have shown that pretrained word embeddings can enhance text representation and improve the generalization capabilities of models. However; the challenge of poor generalization in joint learning models for intent detection and slot filling remains due to limited annotated datasets. Additionally; traditional models face difficulties in capturing both the semantic and syntactic nuances of language; which are vital for accurate intent detection and slot filling. This study proposes a hybridized text representation method using a multichannel convolutional neural network with three embedding channels: non-contextual embeddings for semantic information; part-of-speech (POS) tag embeddings for syntactic features; and contextual embeddings for deeper contextual understanding. Specifically; we utilized word2vec for non-contextual embeddings; one-hot vectors for POS tags; and bidirectional encoder representations from transformers (BERT) for contextual embeddings. These embeddings are processed through a convolutional layer and a shared bidirectional long short-term memory (BiLSTM) network; followed by two softmax functions for intent detection and slot filling. Experiments on the air travel information system (ATIS) and SNIPS datasets demonstrated that our model significantly outperformed the baseline models; achieving an intent accuracy of 97.90% and slot filling F1-score of 98.86% on the ATIS dataset; and an intent accuracy of 98.88% and slot filling F1-score of 97.07% on the SNIPS dataset. These results highlight the effectiveness of our proposed approach in advancing dialogue systems; and paving the way for more accurate and efficient natural language understanding in real-world applications. © (2024); (PeerJ Inc.). All rights reserved;
D O I：
10.7717/PEERJ-CS.2346
中图分类号：
学科分类号：
摘要：
引用
收藏
相关论文
共 50 条
  • [31] A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling
    E, Haihong
    Niu, Peiqing
    Chen, Zhongfu
    Song, Meina
    57TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2019), 2019, : 5467 - 5471
  • [32] An optimised Darknet traffic detection system using modified locally connected CNN-BiLSTM network
    Shaikh, Abdullah Abdul Sattar
    Bhargavi, M. S.
    Kumar, C. Pavan
    INTERNATIONAL JOURNAL OF AD HOC AND UBIQUITOUS COMPUTING, 2023, 43 (02) : 87 - 96
  • [33] Joint Intent Detection and Slot Filling of Knowledge Question Answering for Agricultural Diseases and Pests
    Guo X.
    Hao X.
    Yao X.
    Li L.
    Nongye Jixie Xuebao/Transactions of the Chinese Society for Agricultural Machinery, 2023, 54 (01): : 205 - 215
  • [34] CONVOLUTIONAL NEURAL NETWORK BASED TRIANGULAR CRF FOR JOINT INTENT DETECTION AND SLOT FILLING
    Xu, Puyang
    Sarikaya, Ruhi
    2013 IEEE WORKSHOP ON AUTOMATIC SPEECH RECOGNITION AND UNDERSTANDING (ASRU), 2013, : 78 - 83
  • [35] Joint agricultural intent detection and slot filling based on enhanced heterogeneous attention mechanism
    Hao, Xia
    Wang, Lu
    Zhu, Hongmei
    Guo, Xuchao
    COMPUTERS AND ELECTRONICS IN AGRICULTURE, 2023, 207
  • [36] Sleep Apnea Detection From Variational Mode Decomposed EEG Signal Using a Hybrid CNN-BiLSTM
    Mahmud, Tanvir
    Khan, Ishtiaque Ahmed
    Mahmud, Talha Ibn
    Fattah, Shaikh Anowarul
    Zhu, Wei-Ping
    Ahmad, M. Omair
    IEEE ACCESS, 2021, 9 : 102355 - 102367
  • [37] Earthquake Magnitude Prediction using Spatia-Temporal Features Learning Based on Hybrid CNN-BiLSTM Model
    Kavianpour, Parisa
    Kavianpour, Mohammadreza
    Jahani, Ehsan
    Ramezani, Amin
    Proceedings - 2021 7th International Conference on Signal Processing and Intelligent Systems, ICSPIS 2021, 2021,
  • [38] Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling
    Liu, Bing
    Lane, Ian
    17TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2016), VOLS 1-5: UNDERSTANDING SPEECH PROCESSING IN HUMANS AND MACHINES, 2016, : 685 - 689
  • [39] Towards Explainable Joint Models via Information Theory for Multiple Intent Detection and Slot Filling
    Zhuang, Xianwei
    Cheng, Xuxin
    Zou, Yuexian
    THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 17, 2024, : 19786 - 19794
  • [40] AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling
    Qin, Libo
    Xu, Xiao
    Che, Wanxiang
    Liu, Ting
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, EMNLP 2020, 2020, : 1807 - 1816
12345