Transforming Convolutional Neural Network to an Interpretable Classifier

For machine learning based methods to be accepted in practice, these methods should be either directly or indirectly interpretable or should at least provide a reliable estimate of a confidence degree about their prediction. This is of the importance primarily in critical domains like medicine where the decision should be clearly justified. Although deep neural networks achieve remarkable performance in almost any computer vision task, due to their complexity, their predictions remain difficult to interpret and explain. An interpretable classifier does not only provide the prediction but also a kind of evidence or explanation. In this paper, we present an effective way of transforming a sufficiently well trained convolutional neural network (98.50% accuracy on MNIST dataset, which contains small images of digits) to an interpretable classifier. We use hidden activations of training observations as their descriptors. Next, for every unseen observation, we use these descriptors to identify three most similar training samples based on cosine distance of their hidden activations. Using this approach, we are able not only to predict the class of an unknown observation, but also to justify the prediction by providing three most similar training observations in exchange to a slightly decreased accuracy (98.12%).