Machine-Learning Enabled Biocompatible Capacitive-Electromyographic Bimodal Flexible Sensor for Facial Expression Recognition

Single-mode sensors suffer from poor robustness and insufficient data features in facial expression recognition, so fusing multi-sensor signals is the key to improving the accuracy of expression recognition systems. Here, a biocompatible capacitive-electromyographic dual-mode sensor (CEDS) is presented, consisting of a capacitive pressure sensing unit and dry electrodes for electrophysiological signal monitoring, assembled in a 3D stacking fashion. A double-coupled microstructure is prepared and the electrical double-layer effect is realized by doping ionic liquid, which significantly improves the capacitive performance of the sensor. The application of dry electrodes effectively solves the problems of hydrogel electrodes that are prone to water loss and skin irritation. Besides, the good biocompatibility and antimicrobial properties of CEDS are verified through cytotoxicity and bacteriostatic tests. Based on the sensing of a single signal, a fatigue driving monitoring system and a manipulator control system are constructed respectively. By further integrating the capacitive and electrophysiological signal monitoring functions of CEDS, a 1D convolutional neural network-assisted facial expression recognition system is constructed, which effectively improves the accuracy of expression recognition and demonstrates the great potential of facial expression monitoring systems based on flexible sensor technology in practical applications.