MINING SURPRISING PATTERNS AND THEIR EXPLANATIONS IN CLINICAL DATA

We present a new technique for interactively mining patterns and generating explanations by harnessing the expertise of domain experts. Key to the approach is the distinction between what is unexpected from the perspective of the computational data mining process and what is surprising to the domain experts and interesting relative to their needs. We demonstrate the potential of the approach for discovering patterns and generating rich explanations in a clinical domain. Discovering interesting facts in clinical data is a grand challenge, because medical practitioners and clinicians generally have exceptional knowledge in the problem domain in which they work, however, this knowledge is typically difficult to isolate computationally. To identify the desired surprising patterns, we formally record user knowledge and use that knowledge to filter and constrain the output from an objective data mining technique, with the user making the final judgement about whether a rule is surprising. Specifically, we introduce an unexpectedness algorithm based on association rule mining and Bayesian Networks and a E-explanations technique for explanation generation to identify unexpected patterns. An implemented prototype is successfully demonstrated using a large clinical database recording incidence, prevalance, and outcome of dialysis and kidney transplant patients.