Convolutional-capsule network for gastrointestinal endoscopy image classification

Automated diagnosis of digestive tract diseases from gastrointestinal endoscopy images is of high importance for improving the diagnosis accuracy and efficiency. The current mainstream methods for image classification of digestive tract endoscopy images are based on Convolutional Neural Networks (CNNs). However, due to their inherent defects, CNNs are not strong enough in learning deformation-invariant global features which is essential in gastrointestinal endoscopic image classification. To solve this problem, in this paper we present a two-stage endoscopic image classification method which can effectively combine complementary advantages of midlevel CNN features and a capsule network. Specifically, the core of our method is a lesion-aware CNN feature extraction module which can encode sufficiently detailed information of lesions in midlevel CNN features and in turn enable the subsequent capsule classification network to effectively learn deformation-invariant relationships between image entities. Extensive experiments demonstrate the superiority of our method to the state-of-the-art methods with the classification accuracy of 94.83% on the Kvasir v2 data set and the classification accuracy of 85.99% on the HyperKvasir data set.