Continuous Sign Language Recognition Based on 3D Hand Skeleton Data

In sign language recognition, it is necessary to eliminate the visual problems caused by interference factors such as background and light. Therefore, an end-to-end continuous sign language recognition model is designed using low-redundant skeleton data. Firstly, the shape and trajectory features are extracted from intra frame and inter frame respectively, which can reduce the discreteness of the original samples. Secondly, a series of parallel two-stream residual networks are constructed to fuse shape and trajectory features, further generate the spa-tial-temporal feature sequence. Finally, the attention-based encoder-decoder network is used to realize the map-ping of the fused feature sequence to the translated text. In addition, a new skeleton-based sign language dataset using Leap Motion is collected named LMSLR. Experimental results on the LMSLR dataset and the public CSL dataset show that the proposed model has higher accuracy and lower computational complexity than most models based on video processing. © 2021, Beijing China Science Journal Publishing Co. Ltd. All right reserved.