Novel deep learning-based computer-aided diagnosis system for predicting inflammatory activity in ulcerative colitis

Background and Aims: Endoscopy is increasingly performed for evaluating patients with ulcerative colitis (UC). However, its diagnostic accuracy is largely affected by the subjectivity of endoscopists' experience and scoring methods, and scoring of selected endoscopic images cannot reflect the inflammation of the entire intestine. We aimed to develop an automatic scoring system using deep-learning technology for consistent and objective scoring of endoscopic images and full-length endoscopic videos of patients with UC. Methods: We collected 5875 endoscopic images and 20 full-length videos from 332 patients with UC who under-went colonoscopy between January 2017 and March 2021. We trained the artificial intelligence (AI) scoring system using these images, which was then used for full-length video scoring. To more accurately assess and visualize the full-length intestinal inflammation, we divided the large intestine into a fixed number of "areas" (cecum, 20; trans-verse colon, 20; descending colon, 20; sigmoid colon, 15; rectum, 10). The scoring system automatically scored inflammatory severity of 85 areas from every video and generated a visualized result of full-length intestinal inflam-matory activity. Results: Compared with endoscopist scoring, the trained convolutional neural network achieved 86.54% accu-racy in the Mayo-scored task, whereas the kappa coefficient was .813 (95% confidence interval [CI], .782-.844). The metrics of the Ulcerative Colitis Endoscopic Index of Severity-scored task were encouraging, with accuracies of 90.7%, 84.6%, and 77.7% and kappa coefficients of .822 (95% CI, .788-.855), .784 (95% CI, .744-.823), and .702 (95% CI, .612-.793) for vascular pattern, erosions and ulcers, and bleeding, respectively. The AI scoring system predicted each bowel segment's score and displayed distribution of inflammatory activity in the entire large in-testine using a 2-dimensional colorized image. Conclusions: We established a novel deep learning-based scoring system to evaluate endoscopic images from patients with UC, which can also accurately describe the severity and distribution of inflammatory activity through full-length intestinal endoscopic videos.