Sim-To-Real Transfer Learning Approach for Tracking Multi-DOF Ankle Motions Using Soft Strain Sensors

A data-driven approach has recently been investigated for identifying human joint angles by means of soft strain sensors because of the corresponding modeling difficulty. However, this approach commonly incurs a high computational burden due to the voluminous amount of data required and the time-series-oriented network architecture. Moreover, the nature of soft sensors makes the problem worse due to the inherent nonlinearity and hysteresis of the material. In this study, we developed a novel wearable sensing brace design for measuring multiple degrees of freedom (DOF) ankle motions to minimize hysteresis and to improve the measurement repeatability and developed a computationally efficient calibration method based on sim-to-real transfer learning. By attaching the soft sensors to shin links rather than directly to the ankle joint, the effects of external disturbances during joint motions were minimized. To calibrate the sensors to body motions, transfer learning was used based on the results from musculoskeletal simulation(OpenSim) and sensor data. The average tracking error for ankle motions using the proposed method was found to be 12.0 degrees for five healthy subjects, while the direct deep neural network approach showed an error of 17.9 degrees. The proposed method could be used to calibrate the soft sensors with 1000 times faster training speed while maintaining comparable tracking accuracy with a smaller amount of data.