Evaluating MetaMap's text-to-concept mapping performance

Natural Language Processing (NLP) systems used in the analysis of medical texts are mostly evaluated both as part of a larger system and with respect to a specific goal (document retrieval, decision support). Since NLP systems can be integrated as a module in different applications, an application independent analysis and evaluation of their performance is more tailored to the needs of the application developer. We evaluated the performance of the MetaMap system [1] in mapping words in MEDLINE (R) abstracts to medical concepts present in the UMLS (R) Metathesaurus (R). More specifically, the effects of the distinct user options MetaMap provides are analyzed.