Neurophysiological data augmentation for EEG-fNIRS multimodal features based on a denoising diffusion probabilistic model

Background and objective: The hybrid brain computer interfaces (BCI) combining electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) have attracted extensive attention for overcoming the decoding limitations of the single-modality BCI. With the deepening application of deep learning approaches in BCI systems, its significant performance improvement has become apparent. However, the scarcity of brain signal data limits the performance of deep learning models. Methods: In this study, we proposed an EEG-fNIRS data augmentation framework based on the combination of denoising diffusion probabilistic model (DDPM) and adding Gaussian noise (EFDA-CDG), for enhancing the performance of hybrid BCI systems. Firstly, we unified the temporal and spatial dimensions of EEG and fNIRS by manually extracting features and spatial mapping interpolation to create EEG-fNIRS joint distribution samples. Then, the DDPM generative model was combined with the traditional method of adding Gaussian noise to provide richer training data for the classifier. Finally, we constructed a classification module that applies EEG feature attention and fNIRS terrain attention to improve classification accuracy. Results: In order to evaluate the effectiveness of EFDA-CDG framework, experiments were conducted and fully validated on three publicly available databases and one self-collected database. In the context of a participantdependent training approach, our method achieves accuracy rates of 82.02% for motor imagery, 91.93% for mental arithmetic, and 90.54% for n-back tasks on public databases. Additionally, our method boasts an accuracy rate of 97.82% for drug addiction discrimination task on the self-collected database. Conclusions: EFDA-CDG framework successfully facilitates data augmentation, thereby enhancing the performance of EEG-fNIRS hybrid BCI systems.