Temporal embedding and spatiotemporal feature selection boost multi-voxel pattern analysis decoding accuracy

Background: In fMRI decoding, temporal embedding of spatial features of the brain allows the incorporation of brain activity dynamics into the multivariate pattern classification process, and provides enriched information about stimulus-specific response patterns and potentially improved prediction accuracy. New method: This study investigates the possibility of enhancing the classification performance by exploring temporal embedding, to identify the optimum combination of spatiotemporal features based on their classification performance. We investigated the importance of spatiotemporal feature selection using a slow event-related design adapted from the classic Haxby study (Haxby et al., 2001). Data were collected using a multiband fMRI sequence with temporal resolution of 0.568 s. Comparison with existing methods: A wide range of spatiotemporal observations were created as various combinations of spatiotemporal features. Using both random forest, and support vector machine, classifiers prediction accuracies for these combinations were then compared with the single spatial multivariate pattern approach that uses only a single temporal observation. Results: Our findings showed that, on average, spatiotemporal feature selection improved prediction accuracy. Moreover, the random forest algorithm outperformed the support vector machine and benefitted from temporal information to a greater extent. Conclusions: As expected, the most influential temporal durations were found to be around the peak of the hemodynamic response function, a few seconds after the stimuli onset until - 4 s after the peak of the hemodynamic response function. The superiority of spatiotemporal feature selection over single time-point spatial approaches invites future work to design optimal approaches that incorporate spatiotemporal dependencies into feature selection for decoding.