Convolutional Neural Network-Aided DP-64 QAM Coherent Optical Communication Systems

Optical nonlinearity impairments have been a major obstacle for high-speed, long-haul and large-capacity optical transmission. In this paper, we propose a novel convolutional neural network (CNN)-based perturbative nonlinearity compensation approach in which we reconstruct a feature map with two channels that rely on first-order perturbation theory and build a classifier and a regressor as a nonlinear equalizer. We experimentally demonstrate the CNN equalizer in 375 km 120-Gbit/s dual-polarization 64-quadrature-amplitude modulation (64-QAM) coherent optical communication systems. We studied the influence of the dropout value and nonlinear activation function on the convergence of the CNN equalizer. We measured the bit-error-ratio (BER) performance with different launched optical powers. When the channel size is 11, the optimum BER for the CNN classifier is 0.0012 with 1 dBm, and for the CNN regressor, it is 0.0020 with 0 dBm; the BER can be lower than the 7% hard decision-forward threshold of 0.0038 from -3 dBm to 3 dBm. When the channel size is 15, the BERs at-4 dBm, 4 dBm and 5 dBm can be lower than 0.0020. The network complexity is also analyzed in this paper. Compared with perturbative nonlinearity compensation using a fully connected neural network (2392-64-64), we can verify that the time complexity is reduced by about 25%, while the space complexity is reduced by about 50%.