Consolidated domain adaptive detection and localization framework for cross-device colonoscopic images

Automatic polyp detection has been proven to be crucial in improving the diagnosis accuracy and re-ducing colorectal cancer mortality during the precancerous stage. However, the performance of deep neural networks may degrade severely when being deployed to polyp data in a distinct domain. This domain distinction can be caused by different scanners, hospitals, or imaging protocols. In this paper, we propose a consolidated domain adaptive detection and localization framework to bridge the domain gap between different colonosopic datasets effectively, consisting of two parts: the pixel-level adaptation and the hierarchical feature-level adaptation. For the pixel-level adaptation part, we propose a Gaus-sian Fourier Domain Adaptation (GFDA) method to sample the matched source and target image pairs from Gaussian distributions then unify their styles via the low-level spectrum replacement, which can reduce the domain discrepancy of the cross-device polyp datasets in appearance level without distorting their contents. The hierarchical feature-level adaptation part comprising a Hierarchical Attentive Adap-tation (HAA) module to minimize the domain discrepancy in high-level semantics and an Iconic Con-centrative Adaptation (ICA) module to perform reliable instance alignment. These two modules are reg-ularized by a Generalized Consistency Regularizer (GCR) for maintaining the consistency of their do-main predictions. We further extend our framework to the polyp localization task and present a Cen-tre Besiegement (CB) loss for better location optimization. Experimental results show that our frame-work outperforms other domain adaptation detectors by a large margin in the detection task meanwhile achieves the state-of-the-art recall rate of 87.5% in the localization task. The source code is available at https://github.com/CityU- AIM-Group/ConsolidatedPolypDA . (c) 2021 Elsevier B.V. All rights reserved.