CNN-based camera motion classification using HSI color model for compressed videos

This paper presents a novel camera motion classification framework based on modeling the compressed domain block motion vectors using the HSI color model. The input to the proposed method is the interframe block motion vectors decoded from the compressed bitstream. The block motion vector’s magnitude and orientation are estimated, followed by assigning motion vector orientation to Hue, motion vector magnitude to Saturation, and keeping Intensity at a fixed value. The HSI assignment is then converted into an RGB image followed by supervised learning utilizing a convolutional neural network to recognize eleven camera motion patterns comprising seven pure camera motion patterns and four mixed camera patterns. The proposed method’s premise is based on posing the camera motion classification problem as a color recognition task. Detailed experimental analysis that includes a comparison with state-of-the-art methods, ablation study, and robustness analysis is carried out utilizing block motion vectors obtained from H.264/AVC encoded videos. Results demonstrate accuracies of over 98 % in recognizing eleven camera patterns for the proposed method.