Dual resolution deep learning network with self-attention mechanism for classification and localisation of colorectal cancer in histopathological images

Aims Microscopic examination is a basic diagnostic technology for colorectal cancer (CRC), but it is very laborious. We developed a dual resolution deep learning network with self-attention mechanism (DRSANet) which combines context and details for CRC binary classification and localisation in whole slide images (WSIs), and as a computer-aided diagnosis (CAD) to improve the sensitivity and specificity of doctors' diagnosis. Methods Representative regions of interest (ROI) of each tissue type were manually delineated in WSIs by pathologists. Based on the same coordinates of centre position, patches were extracted at different magnification levels from the ROI. Specifically, patches from low magnification level contain contextual information, while from high magnification level provide important details. A dual-inputs network was designed to learn context and details simultaneously, and self-attention mechanism was used to selectively learn different positions in the images to enhance the performance. Results In classification task, DRSANet outperformed the benchmark networks which only depended on the high magnification patches on two test set. Furthermore, in localisation task, DRSANet demonstrated a better localisation capability of tumour area in WSI with less areas of misidentification. Conclusions We compared DRSANet with benchmark networks which only use the patches from high magnification level. Experimental results reveal that the performance of DRSANet is better than the benchmark networks. Both context and details should be considered in deep learning method.