Self-supervised scientific document recommendation based on contrastive learning

Scientific document recommendation aims to recommend scientific documents that have similar content to a given target scientific document (e.g., paper or patent, etc.). With the explosive growth in scientific documents, how recommending relevant scientific documents from the massive number of scientific documents has become an extremely challenging problem. Existing unsupervised scientific document recommendation works use generic approaches of text representation learning, ignoring the relationships between paragraphs within scientific documents, which is important for highly logical scientific documents. This paper proposes a self-supervised learning method, coupled text pair embedding (CTPE) model, which captures paragraph relations within scientific documents based on contrastive learning. First, we divide the scientific document into two parts. The two parts from the same document are positive samples, and these from different documents are negative samples. Then, we uncover the paragraph relations by contrasting intra-document and inter-document pairs such that intra pairs have the maximum agreement via a contrastive loss in the document embedding space. Finally, we propose a similarity calculation among document embeddings to achieve scientific document recommendations. We perform experiments on three datasets for one patent and two paper recommendation tasks. The experimental results verify the effectiveness of the proposed model. The proposed model can help researchers to efficiently discover relevant literature, foster interdisciplinary connections, and guide their research efforts in the scientometrics community. (The code is available at https://github.com/aitsc/text-representation.)