Low-resource knowledge graph completion based on knowledge distillation driven by large language models

Knowledge graph completion (KGC) refines the existing knowledge graph (KG) by predicting missing entities or relations. Existing methods are mainly based on embeddings or texts but only perform better with abundant labeled data. Hence, KGC in resource-constrained settings is a significant problem, which faces challenges of data imbalance across relations and lack of relation label semantics. Considering that Large Language Models (LLMs) demonstrate powerful reasoning and generation capabilities, this work proposes an LLM-driven Knowledge Graph Completion Distillation (KGCD) model to address low-resource KGC. A two-stage framework is developed, involving teacher-student distillation by using LLM to improve reasoning, followed by fine-tuning on real-world low-resource datasets. To deal with data imbalance, a hybrid prompt design for LLM is proposed, which includes rethink and open prompts. Furthermore, a virtual relation label generation strategy enhances the model's understanding of triples. Extensive experiments on three benchmarks have shown that KGCD's effectiveness for low-resource KGC, achieving improvements in Mean Reciprocal Rank (MRR) by 11% and Hits@1 by 10% on the WN18, MRR by 10% and Hits@1 by 14% on the WN18RR, and MRR by 12% and Hits@1 by 11% on the YAGO3-10. © 2024 Elsevier B.V.