Chinese CNER Combined with Multi-head Self-attention and BiLSTM-CRF

被引：0
|
作者：
Luo X. [1 ,2 ]
Xia X. [2 ]
An Y. [1 ]
Chen X. [1 ]
机构：
[1] Big Data Institute, Central South University, Changsha
[2] Key Laboratory of Network Crime Investigation of Hunan Provincial Colleges, Hunan Police Academy, Changsha
来源：
Hunan Daxue Xuebao/Journal of Hunan University Natural Sciences | 2021年 / 48卷 / 04期
基金：
国家重点研发计划;
关键词：
Chinese electronic medical record; Long short-term memory; Multi-head self-attention; Named entity recognition;
D O I：
10.16339/j.cnki.hdxbzkb.2021.04.006
中图分类号：
学科分类号：
摘要：
Named entity is the main carrier of relevant medical knowledge in Electronic Medical Records (EMRs), so clinical named entity recognition(CNER) has become one of the basic and crucial tasks of clinical text analysis and processing. Due to the particularity of medical text structure and Chinese language, the recognition of clinical named entities for Chinese EMRs still faces great challenges. In this paper, a Chinese clinical named entity recognition method based on multi-head self-attention neural network is proposed. In this method, a character-level feature representation method combined with a domain dictionary is presented. Moreover, based on the BiLSTM-CRF model, a multi-head self-attention mechanism is incorporated to accurately capture the multiple features from different aspects, such as dependency weights between characters and contextual semantic relationships, thereby effectively improving the ability of Chinese clinical named entity recognition. Experimental results demonstrate that the proposed method outperforms other existing methods and has the best recognition performance. © 2021, Editorial Department of Journal of Hunan University. All right reserved.
引用
收藏
页码：45 / 55
页数：10
相关论文
共 19 条
  • [11] OUYANG E, LI Y, JIN L, Et al., Exploring n-gram character presentation in bidirectional RNN-CRF for Chinese clinical named entity recognition, Proceedings of the Evaluation Task at the China Conference on Knowledge Graph and Semantic Computing, pp. 37-42, (2017)
  • [12] XIA Y, WANG Q., Clinical named entity recognition: ECUST in the CCKS-2017 shared task 2, Proceedings of the Evaluation Task at the China Conference on Knowledge Graph and Semantic Computing, pp. 43-48, (2017)
  • [13] HU J, SHI X, LIU Z, Et al., HITSZ_CNER: a hybrid system for entity recognition from Chinese clinical text, Proceedings of the Evaluation Task at the China Conference on Knowledge Graph and Semantic Computing, pp. 25-30, (2017)
  • [14] WANG Q, ZHOU Y, RUAN T, Et al., Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition, Journal of Biomedical Informatics, 92, (2019)
  • [15] QIU J, ZHOU Y, WANG Q, Et al., Chinese clinical named entity recognition using residual dilated convolutional neural network with conditional random field, IEEE Transactions on Nano Bioscience, 18, 3, pp. 306-315, (2019)
  • [16] TANG B, WANG X, YAN J, Et al., Entity recognition in Chinese clinical text using attention-based CNN-LSTM-CRF, BMC Medical Informatics and Decision Making, 19, 3, (2019)
  • [17] VASWANI A, SHAZEER N, PARMAR N, Et al., Attention is all you need, Proceedings of the 31st Annual Conference on Neural Information Processing Systems, pp. 5998-6008, (2017)
  • [18] VITERBI A., Error bounds for convolutional codes and an asymptotically optimum decoding algorithm, IEEE Transactions on Information Theory, 13, 2, pp. 260-269, (1967)
  • [19] WU J, HU X, ZHAO R, Et al., Clinical named entity recognition via bi-directional LSTM-CRF model, Proceedings of the Evaluation Task at the China Conference on Knowledge Graph and Semantic Computing, pp. 31-36, (2017)
12