Utilizing structural metrics from knowledge graphs to enhance the robustness quantification of large language models

Knowledge graphs (KGs) play a critical role in organizing large stores of unstructured information into structured formats. This structured information is then accessible through SPARQL queries or graph libraries based on their structure. KGs enhance search, power AI systems, and facilitate knowledge discovery across domains. In this research, we explore the capabilities of different large language models (LLMs) like CodeLlama, Mistral, and Vicuna, which are recognized for text generation, in handling textual information tasks for constructing knowledge graphs with structured data. Utilizing these LLMs, we generate class descriptions for all the classes of well-known KGs like DBpedia, YAGO, and Google Knowledge Graph. Using these class descriptions, we have extracted RDF triples and used different preprocessing techniques for better refinement and extraction of the graph triples from the generated result. These extracted triples are used for the graph ontology creation. Highlighting the contribution of LLMs to structured graph formation, our study includes a comparison of the constructed KGs using the three LLMs with the existing Knowledge Graphs. Later, these KGs are evaluated using six structural quality metrics encompassing both class and property-related information crucial for KG formation. Our insights prove valuable for researchers exploring these domains, offering guidance on overcoming challenges and maximizing the potential of large language models in knowledge graph construction, text generation, and text extraction.