Chinese CNER Combined with Multi-head Self-attention and BiLSTM-CRF

被引：0
|
作者：
Luo X. [1 ,2 ]
Xia X. [2 ]
An Y. [1 ]
Chen X. [1 ]
机构：
[1] Big Data Institute, Central South University, Changsha
[2] Key Laboratory of Network Crime Investigation of Hunan Provincial Colleges, Hunan Police Academy, Changsha
来源：
Hunan Daxue Xuebao/Journal of Hunan University Natural Sciences | 2021年 / 48卷 / 04期
基金：
国家重点研发计划;
关键词：
Chinese electronic medical record; Long short-term memory; Multi-head self-attention; Named entity recognition;
D O I：
10.16339/j.cnki.hdxbzkb.2021.04.006
中图分类号：
学科分类号：
摘要：
Named entity is the main carrier of relevant medical knowledge in Electronic Medical Records (EMRs), so clinical named entity recognition(CNER) has become one of the basic and crucial tasks of clinical text analysis and processing. Due to the particularity of medical text structure and Chinese language, the recognition of clinical named entities for Chinese EMRs still faces great challenges. In this paper, a Chinese clinical named entity recognition method based on multi-head self-attention neural network is proposed. In this method, a character-level feature representation method combined with a domain dictionary is presented. Moreover, based on the BiLSTM-CRF model, a multi-head self-attention mechanism is incorporated to accurately capture the multiple features from different aspects, such as dependency weights between characters and contextual semantic relationships, thereby effectively improving the ability of Chinese clinical named entity recognition. Experimental results demonstrate that the proposed method outperforms other existing methods and has the best recognition performance. © 2021, Editorial Department of Journal of Hunan University. All right reserved.
引用
收藏
页码：45 / 55
页数：10
相关论文
共 19 条
  • [1] LAFFERTY J, MCCALLUM A, PEREIRA F CN., Conditional random fields: Probabilistic models for segmenting and labeling sequence data, Proceedings of the Eighteenth International Conference on Machine Learning (ICML 2001), pp. 282-289, (2001)
  • [2] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, Et al., Neural architectures for named entity recognition, Proceedings of the 15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 260-270, (2016)
  • [3] DUAN H, ZHENG Y., A study on features of the CRFs-based Chinese named entity recognition, International Journal of Advanced Intelligence Paradigms, 3, 2, pp. 287-294, (2011)
  • [4] ZHOU G D, SU J., Named entity recognition using an HMM-based chunk tagger, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 473-480, (2001)
  • [5] AMARAPPA S, SATHYANARAYANA S V., Kannada named entity recognition and classification using support vector machine, Transactions on Machine Learning and Artificial Intelligence, 5, 1, pp. 43-43, (2017)
  • [6] MCCALLUM A, LI W., Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons, Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, 4, pp. 188-191, (2003)
  • [7] HUANG Z, XU W, YU K., Bidirectional LSTM-CRF models for sequence tagging, Computer Science, (2015)
  • [8] ZHANG J F, BI Z S, WANG J, Et al., Design of Chinese domain named entity recognition framework based on BLSTM-CRF, Journal of Hunan University(Natural Sciences), 46, 3, pp. 117-121, (2019)
  • [9] MA X, HOVY E., End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF, Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, pp. 1064-1074, (2016)
  • [10] LI Z, ZHANG Q, LIU Y, Et al., Recurrent neural networks with specialized word embedding for Chinese clinical named entity recognition, Proceedings of the Evaluation Task at the China Conference on Knowledge Graph and Semantic Computing, pp. 55-60, (2017)
12