Deep Knowledge Distillation Learning for Efficient Wearable Data Mining on the Edge

Wearable data mining on the edge is essential for consumer electronics and providing real-time insights to the users. However, the challenge lies in the high computing complexity and energy consumption of the algorithms, especially the advanced deep neural networks. Focusing on this challenge, we in this research have proposed a deep knowledge distillation learning algorithm to achieve a light-weight edgedeployable deep learning model. We take a special interest in the Electrocardiogram-based cardiac disease detection application. More specifically, we have firstly designed a heavy teacher model for cardiac disease detection. We then have leveraged the soft target distribution of the teacher model to supervise the training of a lightweight student model. In such a way, the student model can learn the knowledge from the teacher model, with an energy-efficient structure that has significantly less parameters. Evaluated on the cardiac disease detection task, our framework has demonstrated promising effectiveness, and this study will therefore greatly advance efficient wearable data mining on the edge in consumer electronics.