Evaluation of a new dataset for visual detection of cervical precancerous lesions

Automated visual evaluation (AVE) is an emerging method to detect and diagnose cervical precancerous lesions by imaging and analysis via a deep learning classifier. Challenges in AVE development come from not only the limited data available, but also a proper design of the learning protocol. The most analyzed dataset (PEG) is traced to a clinical trial at a single site, where all the images were captured in a well controlled environment. Recently, cervical images have been captured by a light-weight mobile solution where the screening images were collected from a wider user pool at many sites. This paper introduces a new data resource (EVA dataset), collected by providers using a mobile colposcope during their routine practice. Compared to PEG, EVA images contain higher levels of data variations and exhibits a different distribution over multiple image attributes including image sharpness, brightness and colorfulness. In order to evaluate the practical value of EVA dataset for cervical high-grade squamous intraepithelial lesions (SIL) diagnosis, we further present an analysis of how a deep learning based framework can be used with both datasets by evaluating three key technical components: (1) region-of-interest (ROI) detection, (2) data augmentation and (3) pre-trained deep learning model selection. Our results indicate that ROI detection and shallow deep learning models usually help the detection on both datasets. While most data augmentations are effective on the EVA dataset, the improvement is less pronounced on PEG. Overall, using a deep-learning based framework looks promising for high-grade SIL diagnosis but there is still large room for improvements, especially on the EVA dataset. The noted differences indicate that the EVA dataset presents more practical challenges for high-grade SIL diagnosis and AVE classifier development.