Un-Apriori: a Novel Association Rule Mining Algorithm for Unstructured EMRs

With the rapid development of hospital information technologies, more and more hospitals build electronic medical record (EMR) systems, which provides a comprehensive source for medical data mining and analysis. Most current EMR systems adopt a mixed structure. On the other hand, most data mining algorithms are designed for highly structured data. In this paper, we study the problem of interesting association rule mining on unstructured EMR data. We first analyze characteristics of unstructured EMR data by analyzing real in-patient department EMR of chronic occupational disease from a major hospital in Qingdao, and then put forward a novel association rule mining algorithm, called Un-Apriori, based on Apriori. Un-Apriori adopts a two-step preprocessing procedure for the original EMR data to resolve problems induced by the unstructured EMR data and satisfy specific requirement of Apriori algorithm. Taking asbestosis, a typical chronic occupational disease, as example we evaluate performance of the proposed Un-Apriori algorithm. Two types of experiments are implemented on the Hadoop platform, results and corresponding analysis are also provided. Experiments show that it is feasible and effective to apply data mining algorithms to discover latent association relationships or patterns hidden in unstructured EMRs so long as the reprocessing method is appropriate. Achievements obtained in this paper also provide valuable assistance decision supports for personalized therapeutic regimens and daily health managements of asbestosis patients.