Identifying Named Entities of Chinese Electronic Medical Records Based on RoBERTa-wwm Dynamic Fusion Model

被引：0
|
作者：
Yunqiu Z. [1 ]
Yang W. [1 ]
Bocheng L. [1 ]
机构：
[1] School of Public Health, Jilin University, Changchun
来源：
Data Analysis and Knowledge Discovery | 2022年 / 6卷 / 2-3期
关键词：
Dynamic Fusion; Electronic Medical Record; Named Entity Recognition; RoBERTa-wwm;
D O I：
10.11925/infotech.2096-3467.2021.0951
中图分类号：
学科分类号：
摘要：
[Objective] This paper proposes an entity recognition model based on RoBERTa-wwm dynamic fusion, aiming to improve the entity identification of Chinese electronic medical records. [Methods] First, we merged the semantic representations generated by each Transformer layer of the pre-trained language model RoBERTa-wwm. Then, we input the bi-directional long short-term memory network and the conditional random field module to recognize the entities of the electronic medical records. [Results] We examined our new model with the dataset of“2017 National Knowledge Graph and Semantic Computing Conference (CCKS 2017)”and self-annotated electronic medical records. Their F1 values reached 94.08% and 90.08%, which were 0.23% and 0.39% higher than the RoBERTa-wwm-BiLSTM-CRF model. [Limitations] The RoBERTa-wwm used in this paper completed the pre-training process with non-medical corpus. [Conclusions] The proposed method could improve the results of entity recognition tasks. © 2022, Chinese Academy of Sciences. All rights reserved.
引用
收藏
页码：242 / 250
页数：8
相关论文
共 20 条
  • [1] Shen L, Li Q, Wang W, Et al., Treatment Patterns and Direct Medical Costs of Metastatic Colorectal Cancer Patients: A Retrospective Study of Electronic Medical Records from Urban China, Journal of Medical Economics, 23, 5, pp. 456-463, (2020)
  • [2] Liu Liu, Wang Dongbo, A Review on Named Entity Recognition, Journal of the China Society for Scientific and Technical Information, 37, 3, pp. 329-340, (2018)
  • [3] McCallum A, Li W., Early Results for Named Entity Recognition with Conditional Random Fields, Feature Induction and Web-Enhanced Lexicons, Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL 2003, pp. 188-191, (2003)
  • [4] Huang Han, Wang Hongyu, Wang Xiaoguang, Automatic Recognizing Legal Terminologies with Active Learning and Conditional Random Field Model, Data Analysis and Knowledge Discovery, 3, 6, pp. 66-74, (2019)
  • [5] Zhao S J., Named Entity Recognition in Biomedical Texts Using an HMM Model[C], Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications-JNLPBA ’04, pp. 87-90, (2004)
  • [6] Feng Jing, Li Zhengwu, Zhang Dengyun, Et al., Named Entity Recognition of Bridge Detection Text Based on Hidden Markov Model, TranspoWorld, 8, pp. 32-33, (2020)
  • [7] Kazama J, Makino T, Ohta Y, Et al., Tuning Support Vector Machines for Biomedical Named Entity Recognition, Proceedings of the ACL-02 Workshop on Natural Language Processing in the Biomedical Domain, pp. 1-8, (2002)
  • [8] Yan Lei, Zhou Lanjiang, Zhang Jianan, Et al., Lao Organization Name Entity Recognition Method Fusing Multiple Features, Modern Electronics Technique, 43, 19, pp. 122-125, (2020)
  • [9] Cocos A, Fiks A G, Masino A J., Deep Learning for Pharmacovigilance: Recurrent Neural Network Architectures for Labeling Adverse Drug Reactions in Twitter Posts, Journal of the American Medical Informatics Association, 24, 4, pp. 813-821, (2017)
  • [10] Ji B, Liu R, Li S S, Et al., A BiLSTM-CRF Method to Chinese Electronic Medical Record Named Entity Recognition, Proceedings of the 2018 International Conference on Algorithms, Computing and Artificial Intelligence, pp. 1-6, (2018)
12