Predicting the diabetic foot in the population of type 2 diabetes mellitus from tongue images and clinical information using multi-modal deep learning

Aims Based on the quantitative and qualitative fusion data of traditional Chinese medicine (TCM) and Western medicine, a diabetic foot (DF) prediction model was established through combining the objectified parameters of TCM and Western medicine.Methods The ResNet-50 deep neural network (DNN) was used to extract depth features of tongue demonstration, and then a fully connected layer (FCL) was used for feature extraction to obtain aggregate features. Finally, a non-invasive DF prediction model based on tongue features was realized.Results Among the 391 patients included, there were 267 DF patients, with their BMI (25.2 vs. 24.2) and waist-to-hip ratio (0.953 vs. 0.941) higher than those of type 2 diabetes mellitus (T2DM) group. The diabetes (15 years vs. 8 years) and hypertension durations (10 years vs. 7.5 years) in DF patients were significantly higher than those in T2DM group. Moreover, the plantar hardness in DF patients was higher than that in T2DM patients. The accuracy and sensitivity of the multi-mode DF prediction model reached 0.95 and 0.9286, respectively.Conclusion We established a DF prediction model based on clinical features and objectified tongue color, which showed the unique advantages and important role of objectified tongue demonstration in the DF risk prediction, thus further proving the scientific nature of TCM tongue diagnosis. Based on the qualitative and quantitative fusion data, we combined tongue images with DF indicators to establish a multi-mode DF prediction model, in which tongue demonstration and objectified foot data can correct the subjectivity of prior knowledge. The successful establishment of the feature fusion diagnosis model can demonstrate the clinical practical value of objectified tongue demonstration. According to the results, the model had better performance to distinguish between T2DM and DF, and by comparing the performance of the model with and without tongue images, it was found that the model with tongue images performed better.