Machine learning compensation of fiber nonlinear noise

This paper extends previous studies work on the application of Machine Learning (ML) to distortion compensation in optical communication systems with optical fibers nonlinear coefficients (gamma) that exceed current industry standards e.g. gamma > 1.4W(-1) km(-1). To quantify the improvement afforded by ML methods under different transmission conditions the procedures are applied to a model of a typical single-frequency optical communication system with, a 3200 km fiber length, double polarization, and a 16-QAM modulation format. The performance of both transmitters and receivers that incorporate Neural Networks (NNs) are then examined over a wide range of gamma values. In all cases considered, the system Q-factor is improved although the degree of enhancement is dependent on the signal to noise ratio. The ML structures that were investigated include Siamese neural networks (SNN) implemented at the receiver end as well as two-stage architectures that employ NNs at the transmitter together with a classifier at the receiver side. Classifiers ranging from simple decision tree structures to boosting, forests, extra trees, and Multi-layer perceptron (MLP) were further examined and found to provide significant enhancement for gamma > 4W(-1)km(-1). The optimal performance for highly nonlinear systems was achieved for two-stage systems with random forest or extra tree classifiers. Finally, empirical equations for each ML technique were derived that relate the Q-factor enhancement and the value of gamma, and the number of triplet terms that are input into the neural network. These results could potentially be employed to relax manufacturing constraints and accordingly reduce system costs.