Language Models as SPARQL Query Filtering for Improving the Quality of Multilingual Question Answering over Knowledge Graphs

Question Answering systems working over Knowledge Graphs (KGQA) generate a ranked list of SPARQL query candidates for a given natural-language question. In this paper, we follow our long-term research agenda of providing trustworthy KGQA systems - here - by presenting a query filtering approach that utilizes (large) language models (LMs/LLMs), s.t., correct and incorrect queries can be distinguished. In contrast to the previous work, we address here multilingual questions represented in major languages (English, German, French, Spanish, and Russian), and confirm the generalizability of our approach by also evaluating it on low-resource languages (Ukrainian, Armenian, Lithuanian, Belarusian, and Bashkir). For our experiments, we used the following LMs: BERT, DistilBERT, Mistral, Zephyr, GPT-3.5, and GPT-4. The LMs were applied to the KGQA systems - QAnswer and MemQA - as SPARQL query filters. The approach was evaluated on the multilingual Wikidata-based dataset QALD-9-plus. The experimental results suggest that the KGQA systems achieve quality improvements for all languages when using our query-filtering approach.