TG-Dance: TransGAN-Based Intelligent Dance Generation with Music

Intelligent choreographic from music is a popular field of study currently. Many works use fragment splicing to generate new motions, which lacks motion diversity. When the input is only music, the frame-by-frame generation methods lead to similar motions generated by the same music. Some works improve this problem by adding motions as one of the inputs, but requires a high number of frames. In this paper, a new transformer-based neural network, TG-dance, is proposed for predicting high-quality 3D dance motions that follow the musical rhythms. We propose a new idea of multi-level expansion of motion sequences and design a new motion encoder, using a multi-level transformer-upsampling layer. The multi-head attention in the transformer allows better access to contextual information. The upsampling can greatly reduce motion frames input, and is memory friendly. We use generative adversarial network to effectively improve the quality of generated motions. We designed experiments on the publicly available large dataset AIST++. The experimental results show that TG-dance network outperforms the latest models in quantitative and qualitative. Our model inputs fewer frames of motion sequences and audio features to predict high-quality 3D dance motion sequences that follow the rhythm of the music.