Nonlinear Cross-View Sample Enrichment for Action Recognition

Advanced action recognition methods are prone to limited generalization performances when trained on insufficient amount of data. This limitation results from the high expense to label training samples and their insufficiency to capture enough variability due to viewpoint changes. In this paper, we propose a solution that enriches training data by transferring their features across views. The proposed method is motivated by the fact that cross-view features of the same actions are highly correlated. First, we use kernel-based canonical correlation analysis (CCA) to learn nonlinear feature mappings that take multi-view data from their original feature spaces into a common latent space. Then, we transfer training samples from source to target views by back-projecting their CCA features from latent to view-dependent spaces. We experiment this cross-view sample enrichment process for action classification and we study the impact of several factors including kernel choices as well as the dimensionality of the latent spaces.