Attempting to Use MetaMap in Clinical Practice: A Feasibility Study on the Identification of Medical Concepts from Italian Clinical Notes

This study assessed the feasibility of using MetaMap to identify medical concepts from clinical notes written in Italian. We performed two experiments: in "EXP 1", we used MetaMap to annotate Italian texts using a knowledge source consisting of Italian UMLS sources only; in "EXP 2", we used MetaMap to analyze an English unsupervised translated version of the original Italian texts. We considered medical concepts related to three semantic categories: "Disorders", "Findings" and "Symptoms". Average recall, precision and F-measure were equal to 0.53, 0.98 and 0.69 in "EXP 1", and to 0.75, 0.95 and 0.83 in "EXP 2". For both "EXP 1" and "EXP 2" MetaMap showed better performances for the "Disorders" than for "Findings" and "Symptoms". In conclusion, when using MetaMap with the English translation of the Italian clinical notes, we obtained performances good enough to allow using MetaMap in clinical practice. Further investigation about the types of MetaMap's failures could be useful to understand how to improve performances even better.