Joint intent detection and slot filling with syntactic and semantic features using multichannel CNN-BiLSTM

被引：0
|
作者：
Muhammad, Yusuf Idris [1 ]
Salim, Naomie [1 ]
Zainal, Anazida [1 ]
机构：
[1] Faculty of Computing, Universiti Teknologi Malaysia, Johor, Skudai, Malaysia
来源：
PeerJ Computer Science | 2024年 / 10卷
关键词：
Understanding spoken language is crucial for conversational agents; with intent detection and slot filling being the primary tasks in natural language understanding (NLU). Enhancing the NLU tasks can lead to an accurate and efficient virtual assistant thereby reducing the need for human intervention and expanding their applicability in other domains. Traditionally; these tasks have been addressed individually; but recent studies have highlighted their interconnection; suggesting better results when solved together. Recent advances in natural language processing have shown that pretrained word embeddings can enhance text representation and improve the generalization capabilities of models. However; the challenge of poor generalization in joint learning models for intent detection and slot filling remains due to limited annotated datasets. Additionally; traditional models face difficulties in capturing both the semantic and syntactic nuances of language; which are vital for accurate intent detection and slot filling. This study proposes a hybridized text representation method using a multichannel convolutional neural network with three embedding channels: non-contextual embeddings for semantic information; part-of-speech (POS) tag embeddings for syntactic features; and contextual embeddings for deeper contextual understanding. Specifically; we utilized word2vec for non-contextual embeddings; one-hot vectors for POS tags; and bidirectional encoder representations from transformers (BERT) for contextual embeddings. These embeddings are processed through a convolutional layer and a shared bidirectional long short-term memory (BiLSTM) network; followed by two softmax functions for intent detection and slot filling. Experiments on the air travel information system (ATIS) and SNIPS datasets demonstrated that our model significantly outperformed the baseline models; achieving an intent accuracy of 97.90% and slot filling F1-score of 98.86% on the ATIS dataset; and an intent accuracy of 98.88% and slot filling F1-score of 97.07% on the SNIPS dataset. These results highlight the effectiveness of our proposed approach in advancing dialogue systems; and paving the way for more accurate and efficient natural language understanding in real-world applications. © (2024); (PeerJ Inc.). All rights reserved;
D O I：
10.7717/PEERJ-CS.2346
中图分类号：
学科分类号：
摘要：
引用
收藏
相关论文
共 50 条
  • [1] Joint Intent Detection and Slot Filling via CNN-LSTM-CRF
    Kane, Bamba
    Rossi, Fabio
    Guinaudeau, Ophelie
    Chiesa, Valeria
    Quenel, Ilhem
    Chau, Stephan
    2020 6TH IEEE CONGRESS ON INFORMATION SCIENCE AND TECHNOLOGY (IEEE CIST'20), 2020, : 342 - 347
  • [2] Encoding Syntactic Knowledge in Transformer Encoder for Intent Detection and Slot Filling
    Wang, Jixuan
    Wei, Kai
    Radfar, Martin
    Zhang, Weiwei
    Chung, Clement
    THIRTY-FIFTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THIRTY-THIRD CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE AND THE ELEVENTH SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2021, 35 : 13943 - 13951
  • [3] Joint model of intent detection and slot filling of knowledge question for crop diseases and pests using CNN-Transformer
    Wang, Lu
    Liu, Ruilin
    Huang, Jingzhong
    Guo, Xuchao
    Nongye Gongcheng Xuebao/Transactions of the Chinese Society of Agricultural Engineering, 2024, 40 (13): : 156 - 162
  • [4] WAIS: Word Attention for Joint Intent Detection and Slot Filling
    Chen, Sixuan
    Yu, Shuai
    THIRTY-THIRD AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE / THIRTY-FIRST INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE / NINTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE, 2019, : 9927 - 9928
  • [5] Dirichlet variational autoencoder for joint slot filling and intent detection
    Gao, Wang
    Wang, Yu-Wei
    Zhang, Fan
    Fang, Yuan
    Journal of Computers (Taiwan), 2021, 32 (02) : 61 - 73
  • [6] A hybrid 1D CNN-BiLSTM model for epileptic seizure detection using multichannel EEG feature fusion
    Ravi, Swathy
    Radhakrishnan, Ashalatha
    BIOMEDICAL PHYSICS & ENGINEERING EXPRESS, 2024, 10 (03)
  • [7] ELSF: Entity-Level Slot Filling Framework for Joint Multiple Intent Detection and Slot Filling
    Zhu, Zhanbiao
    Huang, Peijie
    Huang, Haojing
    Xu, Yuhong
    Lin, Piyuan
    Lao, Leyi
    Chen, Shaoshen
    Xie, Haojie
    Yin, Shangjian
    IEEE/ACM Transactions on Audio Speech and Language Processing, 2024, 32 : 4880 - 4893
  • [8] Joint intent detection and slot filling using weighted finite state transducer and BERT
    Abro, Waheed Ahmed
    Qi, Guilin
    Aamir, Muhammad
    Ali, Zafar
    APPLIED INTELLIGENCE, 2022, 52 (15) : 17356 - 17370
  • [9] Joint intent detection and slot filling using weighted finite state transducer and BERT
    Waheed Ahmed Abro
    Guilin Qi
    Muhammad Aamir
    Zafar Ali
    Applied Intelligence, 2022, 52 : 17356 - 17370
  • [10] Multitask Learning with Knowledge Base for Joint Intent Detection and Slot Filling
    He, Ting
    Xu, Xiaohong
    Wu, Yating
    Wang, Huazhen
    Chen, Jian
    APPLIED SCIENCES-BASEL, 2021, 11 (11):
12345