Fine-grained and coarse-grained contrastive learning for text classification

Pre-trained language models based on contrastive learning have shown to be effective in text classification. Although its great success, contrastive learning still has shortcomings. First, most of the existing contrastive learning methods neglect to consider linguistic knowledge, which can improve the performance of contrastive learning. Second, hard negatives have been proven to improve the quality of visual representations. However, generating hard negatives still needs to be explored in the field of text classification. In this paper, we propose a novel fine-grained (word-level) and coarse-grained (sentence-level) contrastive learning that can obtain high- quality sentence representations to improve the performance of text classification. Specifically, we construct word-level positive and negative sample pairs by WordNet and perform fine-grained contrastive learning to inject linguistic knowledge. Then we utilize the mixup to generate hard negatives while eliminating false negatives and use coarse-grained contrastive learning to train the model. Experiments on multiple public datasets demonstrate that our method outperforms state-of-the-art methods.