Searching for anomalous polarization modes of the stochastic gravitational wave background with LISA and Taiji

The vector and scalar polarization modes of gravitational waves do not exist in general relativity, and their detection would have significant impacts on fundamental physics. In this paper, we explored the detectability of these anomalous polarization modes in a gravitational wave background around 1 mHz with the future LISA-Taiji network. The inherent geometrical symmetry of the network largely simplifies the correlation analysis. By taking a suitable linear combination of the correlated outputs, the contribution of the standard tensor modes can be canceled algebraically, and the anomalous modes can be exclusively examined. We provide concise expressions for the signal-to-noise ratios of the anomalous modes with this cancellation method. We also discuss the possibility of separately estimating the amplitudes of the vector and scalar modes, using the overall frequency dependence of the associated overlap reduction functions.