Intra-Channel Nonlinearity Mitigation in Optical Fiber Transmission Systems Using Perturbation-Based Neural Network

In this work, a perturbation-based neural network (P-NN) scheme with an embedded bidirectional long short-term memory (biLSTM) layer is investigated to compensate for the Kerr fiber nonlinearity in optical fiber communication systems. Numerical simulations have been carried out in a 32-Gbaud dual-polarization 16-ary quadrature amplitude modulation (DP-16QAM) transmission system. It is shown that this P-NN equalizer can achieve signal-to-noise ratio improvements of similar to 1.37 dB and similar to 0.80 dB, compared to the use of a linear equalizer and a single step per span (StPS) digital back propagation (DBP) scheme, respectively. The P-NN equalizer requires lower computational complexity and can effectively compensate for intra-channel nonlinearity. Meanwhile, the performance of P-NN is more robust to the distortion caused by equalization enhanced phase noise (EEPN). Furthermore, it is also found that there exists a tradeoff between the choice of modulation format and the nonlinear equalization schemes for a given transmission distance.