SynHAR: Augmenting Human Activity Recognition With Synthetic Inertial Sensor Data Generated From Human Surface Models

We investigate combined, personalised biomechanical dynamics models and human surface models to synthesise IMU sensor time series data and to improve Human Activity Recognition (HAR) model performance for activities of daily living (ADLs). We analyse two model training scenarios: (1) data fusion of synthetic and measurement IMU data to train HAR models directly, and (2) pretraining HAR models with synthetic and measured IMU data and subsequent transfer learning with public benchmark datasets. Furthermore, we analyse how the synthetic IMU data helps in configurations with scarce measurement data by limiting the number of participants and IMUs in both training scenarios. We evaluate three state-of-the-art HAR models to determine the benefit of our approach. Our results show that the IMU data synthesis approach improves performance across all HAR models in both training scenarios. Depending on the HAR model, synthetic data increased the macroF1 score on average by 8% for configurations with reduced data and by up to 7.5% for transfer learning. In the transfer learning scenario, combining synthetic data with measurement data during pretraining outperformed the results obtained by pretraining with measurement data only, by an average of 5.2% across the public datasets. We conclude that augmenting HAR models with synthetic IMU data provides clear performance improvements for HAR and a versatile approach to accurately reflect human movements.