A Shortcut Enhanced LSTM-GCN Network for Multi-Sensor Based Human Motion Tracking

Multi-sensor based motion tracking is of great interest to the robotics community as it may lessen the need for expensive optical motion capture equipment. However, the traditional convolution algorithms have difficulty adapting to the data due to the changes of joints' relative position during motion. The time-series networks often used in the past ignore the spatial characteristics of sensors. We tackle this challenge by combining long short-term memory (LSTM) with graph convolution network (GCN), adding the prior knowledge of sensor distribution, and integrating it into the motion law through the adjacency matrix. This article proposes a novel shortcut enhanced LSTM-GCN network (SE-LSTM-GCN). It connects LSTM and GCN in sequence and extracts temporal and spatial features of data. At the same time, the shortcut is used in the network to enhance the output of two middle layers and to restore the filtered information. Our experimental results on two different motion tracking datasets show that the proposed network is able to learn the mapping relationship with better universality, less tracking error, and without increasing much training time, and can better perform human motion tracking tasks.