E-FNet: A EEG-fNIRS dual-stream model for Brain-Computer Interfaces

With the rapid advancements in Brain-Computer Interfaces (BCI) and multimodal technology, researchers are actively exploring the intricate domain of multimodal BCI. The integration of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) in BCI systems has garnered significant attention due to the combination of strengths from both modalities and the overcoming of limitations inherent in standalone systems. In this study, the E-FNet architecture is introduced, a (2+1)D dual-stream model explicitly designed to process multimodal EEG and fNIRS data for brain-machine interface applications. By leveraging the (2+1)D architecture and adopting a bias-free design, E-FNet efficiently processes multimodal data. A unified 4D tensor representation enables the achievement of both temporal and spatial alignment of EEG and fNIRS signals. For Motor Imagery (MI) and Mental Arithmetic (MA) tasks, the early integration of EEG and fNIRS signals yields remarkable accuracy rates of 95.86% and 95.80% respectively, outperforming the accuracy obtained with EEG signals alone. Moreover, the minimal variability in results across subjects underscores the complementary effects achieved through the integration of different signals.