Tennis players' hitting action recognition method based on multimodal data

In order to improve the recognition accuracy of hitting movements, a tennis player hitting movement recognition method based on multimodal data is proposed. First, we collect acceleration modal data of hitting movements and extract acceleration characteristics of hitting movements. Then, we collect deep modal data of hitting movements and extract deep optical flow features of hitting movements. Finally, we collect RGB modal images of hitting movements, and use recurrent neural networks to extract RGB features of hitting movements. The canonical correlation analysis method is selected to fuse the acceleration characteristics, depth optical flow characteristics and RGB characteristics of tennis players' hitting movements. The feature fusion result is taken as the input of the spatiotemporal convolutional neural network, and the spatiotemporal convolutional neural network is used to output the tennis player's stroke action recognition result. The experimental results show that this method effectively recognises tennis players' hitting movements, with an accuracy of over 99%.