STAN: Spatio-Temporal Analysis Network for efficient video action recognition

Action recognition, whose goal is identifying and extracting spatio-temporal features from video content, is a foundation of work in video understanding. However, current methods for sampling these features are computationally intensive and necessitate complex model architectures. In this paper, we improve 2D CNNs for action recognition tasks, keeping the model streamlined while making effective and complementary use of spatio-temporal features of videos. We propose a model, the Spatio-Temporal Analysis Network (STAN), which strikes a balance between model complexity and recognition accuracy. It contains two key components that affect spatio-temporal features: Temporal Embedding Head (TEH) and Spatio-Temporal Attention (STA). TEH introduces a differential analysis of actions, allowing the model to capture subtle temporal changes and enhance its representational capabilities. STA offers a novel perspective on video streams, improving the spatiotemporal representation without significantly increasing computational demands. It achieves this through a stylized spatial analysis of the features that differ from the conventional optical flow and depth map methods. The results in four datasets demonstrate our methodology's remarkable efficiency and accuracy. Compared to 3D CNNs, our method improves action recognition accuracy by 1.2% and reduces computational costs by 30%. With a dataset utilization rate of only 20% from UCF101, our model achieves an accuracy of 91.68%.