Visualization for Histopathology Images using Graph Convolutional Neural Networks

With an increase in the use of deep learning for computer-aided diagnosis in medical images, the criticism of the black-box nature of the deep learning models is also on the rise. The medical community prefers interpretable models for its due diligence and advancing the understanding of disease and treatment mechanisms. For instance, in histology, while cells and their spatial relationships manifest in rich detail, it is difficult to modify convolutional neural networks to point out the relevant visual features. We adopt an approach to model the histology of a cancer tissue as a graph of its constituent nuclei. We analyze this graph using two novel graph convolutional network frameworks - one based on node occlusion, and another based on attention mechanism - for disease classification and visualization. The proposed methods highlight the relative contribution of each cell nucleus in the disease diagnosis. As proofs of concept, our frameworks not only distinguish accurately between IDC and DCIS breast cancers as well as Gleason 3 and 4 prostate cancers, but they also highlight important visual details, such as boundaries of tumor nests in DCIS and those of glands in Gleason 3.