Joint Extraction of Clinical Entities and Relations Using Multi-head Selection Method

The extraction of entities and relations from unstructured clinical records has been attracting increasing attention. In addition to the existing traditional methods, deep learning methods have also been proposed for entity and relation extraction. However, previous work on clinical entity and relation extraction did not consider the multiple relations between clinical entities, which often exist in clinical texts. To deal with multiple relations, we propose using a multi-head selection method for clinical entity and relation extraction. As pre-trained language models have been shown to be effective for clinical entity and relation extraction, we integrate a pre-trained language model with a multi-head model to jointly extract clinical entities and relations. The experimental results show that the proposed model is effective for entity and relation extraction on both the i2b2/VA 2010 and n2c2 2018 challenge datasets and outperforms the topranking systems in the n2c2 2018 challenge. We also evaluate the impact of four existing pre-trained language models on clinical entity and relation extraction performance. The domainspecific pre-trained language model improves the performance of clinical entity and relation extraction. Between BERT and CharacterBERT, which uses a Character-CNN module instead of BERT's wordpiece system to represent entire words, we find that BERT outperforms CharacterBERT on joint extraction of clinical entities and relations.