Molecular representation contrastive learning via transformer embedding to graph neural networks

Molecular property prediction has shown great performance using graph neural networks (GNNs). However, due to the lack of expansion potential and the scarcity of available labeling data, GNNs are unable to generate appropriate molecular representation. In this study, we propose MolFG, a new contrastive learning (CL) pre-training framework for predicting molecular properties. Meanwhile, we also propose FormerGraph, an effective molecular graph representation strategy, aiming to devise an effective method for learning information regarding molecular features. After pre-training on 10 million unlabeled molecules and then fine-tuning multiple types of downstream tasks to predict molecular properties. The encouraging results revealed that MolFG could effectively extract meaningful chemical insights to generate interpretable representations and differentiate chemically plausible molecular similarities. On most molecular benchmark datasets, MolFG rivals or surpasses supervised learning methods with sophisticated feature engineering. Compared to the previous best supervised model, MolFG demonstrates an average 7.5% gain in ROC-AUC on 7 classification tasks and a 1.9% decrease in scaled average error on 6 regression tasks. Numerous experimental outcomes on downstream tasks demonstrate that the MolFG model can significantly enhance its effectiveness in predicting molecular properties.