Medical Entity Recognition in Twitter using Conditional Random Fields

Identifying medical entities such as disease, medication, and treatment in a social media text is a big challenge due to the lack of words and the noisy nature of the text. However, social media portal like Twitter is an interesting source with a broad range of topics, especially health news and events. Therefore, detecting medical entities on Twitter can support public health surveillance and early event detection for health news and trends. Thus, we proposed an approach to identify a medical entity using a statistical modeling method called Conditional Random Field (CRF). With a small amount of labeled data, our proposed NER model outperforms the other machine learning model in annotating seven entity types in Twitter data by the F1-score results of 68%. This score is a relatively good score for such a task with limited training data.