A Pre-trained Clinical Language Model for Acute Kidney Injury

Pre-trained contextual language models such as BERT have dramatically improved performances for many NLP tasks recently. However, few have explored BERT on specific medical domain tasks such as early prediction for Acute Kidney Injury (AKI). Since much of the clinical information is contained in clinical notes that are largely unstructured text, in this paper, we present an AKI domain-specific pre-trained language model based on BERT (AKI-BERT) that could be used to mine the clinical notes for AKI early prediction. Our experiments on MIMIC-III dataset demonstrate that AKI-BERT can yield performance improvements for AKI early prediction.