Nonlinearity mitigation of self-phase modulation effect in coherent optical system

In this paper, the proficiency of nonlinearity compensation studied on the polarization division multiplexing 0.3 return-to-zero quadrature phase-shift keying (PDM-0.3RZ-QAM) coherent optical system. The considerable nonlinear effect is the self-phase modulation (SPM) in the presence of chromatic dispersion. The nonlinearity mitigation technique based on the digital backpropagation (DBP) algorithm is implemented on the receiver side. The performance of the system was investigated in different terms as the optical signal-noise ratio, bit error rate, and the maximum transmission distance. The results demonstrated that the penalties were decreased by 3 dB. The maximum achievable distance was extended to 2000 km. © 2022 Walter de Gruyter GmbH, Berlin/Boston 2022.