Pretrained Models with Adversarial Training for Named Entity Recognition in Scientific Text

Named entity recognition (NER) is an important fundamental task in natural language processing (NLP). This paper describes a method for named entity recognition based on pretrained models and adversarial training in scientific text. The scientific entity recognition task requires the model to identify 7 different scientific term entities. There are some issues with a given dataset, such as imbalanced label classes, excessively long entity bounds, and inconsistent entity labeling. To address these issues, we proposed to use focal loss instead of existing cross-entropy loss. Further, we used one of the common adversarial training methods, i.e., Fast Gradient Method (FGM) to perform semi-supervised NER. The experimental results show that our adversarial training method considerably enhances the performance of the model, and the method used in this study achieves the highest F-1-score of 88.92 %. Moreover, our results also prove that SciBERT is better suited to the task of named entity recognition in scientific text and that the focal loss successfully solves the problem of data imbalance.