NAT-NL2GQL: A Novel Multi-Agent Framework for Translating Natural Language to Graph Query Language

Yuanyuan Liang † , Tingyu Xie ‡ , Gan Peng † , Zihao Huang † , Yunshi Lan † , Weining Qian † · 2024

Read NAT-NL2GQL: A Novel Multi-Agent Framework for Translating Natural Language to Graph Query Language by Yuanyuan Liang † , Tingyu Xie ‡ , Gan Peng † , Zihao Huang † , Yunshi Lan † , Weining Qian † (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-10434.txt.

Introduction

Yuanyuan Liang and five co-authors translate plain questions into graph database queries with a team of AI agents, in a 2024 paper. Graph databases store linked data, but writing their query language by hand takes skill.

Their framework, NAT-NL2GQL, splits the job among three agents. One cleans the question and finds the names and paths it mentions, along with the parts of the database schema that matter. A second, a language model trained for the task, writes the query. A third runs the query and fixes errors from the results. The team also notes that training data for this task is scarce. Splitting the work lets each agent check the others, which suits the messiness of real questions.

Catalog record

CREATORYuanyuan Liang † , Tingyu Xie ‡ , Gan Peng † , Zihao Huang † , Yunshi Lan † , Weining Qian †
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-10434

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NAT-NL2GQL: A Novel Multi-Agent Framework for Translating Natural Language to Graph Query Language Yuanyuan Liang † , Tingyu Xie ‡ , Gan Peng † , Zihao Huang † , Yunshi Lan † , Weining Qian † Abstract The emergence of Large Language Models (LLMs) has revolutionized many fields, not only traditional natural language processing (NLP) tasks. Recently, research on applying LLMs to the database field has been booming, and as a typical non-relational database, the use of LLMs in graph database research has naturally gained significant attention. Recent efforts have increasingly focused on leveraging LLMs to translate natural language into graph query language (NL2GQL). Although some progress has been made, these methods have clear limitations, such as their reliance on streamlined processes that often overlook the potential of LLMs to autonomously plan and collaborate with other LLMs in tackling complex NL2GQL challenges. To address this gap, we propose NAT-NL2GQL, a novel multi-agent framework for translating natural language to graph query language. Specifically, our framework consists of three synergistic agents: the Preprocessor agent, the Generator agent, and the Refiner agent. The Preprocessor agent manages data processing as context, including tasks such as name entity recognition, query rewriting, path linking, and the extraction of query-related schemas. The Generator agent is a fine-tuned LLM trained on NL-GQL data, responsible for generating corresponding GQL statements based on queries and their related schemas. The Refiner agent is tasked with refining the GQL or context using error information obtained from the GQL execution results. Given the scarcity of high-quality open-source NL2GQL datasets based on nGQL syntax, we developed StockGQL, a dataset constructed from a financial market graph database, which will be released publicly for future researches. To demonstrate the effectiveness of our proposed NAT-NL2GQL framework, we conducted experiments on the StockGQL and SpCQL datasets. Experimental results reveal that our method significantly outperforms baseline approaches, highlighting its potential for advancing NL2GQL research. The StockGQL dataset can be accessed at: https://github.com/leonyuancode/StockGQL . [email protected], [email protected],{gpeng, zhHuang}@stu.ecnu.edu.cn, {yslan, wnqian}@dase.ecnu.edu.cn I Introduction Graph data is gaining prominence in modern data science due to its ability to uncover complex relationships, enhance information connectivity, and facilitate intelligent decision-making. With its distinctive relational representation and efficient processing capabilities, graph data proves invaluable across diverse fields such as finance, healthcare, and social networks, particularly for managing highly connected and structurally complex data [ 1 , 2 ] . Relational databases (DBs) are purpose-built for the efficient storage and management of structured data, while graph data demands specialized graph DBs to enable effective storage and processing [ 3 ] . These databases are specifically optimized for accessing and querying graph-structured data, facilitating the efficient manipulation of complex relationships. Popular graph databases, including Neo4j, NebulaGraph, and JanusGraph, offer distinct features and applications, empowering users to better analyze and utilize intricate data relationships [ 4 , 5 ] . Despite the growing importance of graph data across various domains, ordinary users often encounter significant challenges when interacting with graph DBs due to their specialized and complex operations. The lack of technical expertise among many individuals prevents them from fully harnessing the potential of graph data, thereby limiting its broader adoption in real-world applications [ 6 ] . Furthermore, the intricate and nuanced syntax of graph query languages (GQL) presents additional barriers, particularly for users attempting to translate natural language (NL) into GQL—a task commonly referred to as NL2GQL. These challenges collectively make NL2GQL a highly demanding problem [ 7 , 8 ] . Figure 1 illustrates an example of NL2GQL, showcasing its key components: natural language understanding, DB schema comprehension, and the generation and execution of GQL. This highlights the critical need for a system capable of automating the NL2GQL process. By lowering the barriers to entry, such a system would empower users to seamlessly perform data queries and analyses, thus promoting the widespread adoption and practical utilization of graph data. Fig. 1 : The demonstration of the NL2GQL task transforming the user’s natural language into a graph query language that can be executed on a graph DB. In many cases, NL2GQL can be viewed as a specialized application of the sequence-to-sequence (Seq2Seq) task. As such, modern approaches have progressed beyond initial template-based methods to focus on generative models for automatically producing GQL. Compared to template-based techniques, generative models offer greater flexibility and accuracy, enabling the handling of more complex query requirements. This advancement effectively meets the evolving demands of application scenarios and rising user expectations. The study [ 6 ] was the first to explore the application of a Seq2Seq framework for NL2GQL, utilizing both the Seq2Seq model and copying mechanism to generate GQL. Although the accuracy of their proposed baseline method is relatively low, just above 2\% , they also introduced a general-domain NL2GQL dataset, SpCQL. Building on the similarities between GQL and SQL, [ 9 ] developed a SQL2Cypher algorithm designed to map SQL queries to Cypher queries. However, this algorithm is limited to constructing an NL2GQL dataset. Furthermore, the substantial differences between GQL and SQL present significant challenges in transferring Text2SQL methods to NL2GQL. The study [ 10 ] proposes the CoBGT model, which combines BERT, GraphSAGE [ 11 ] , and Transformer, to perform key-value extraction, relation-property prediction, and Cypher query generation. The work [ 12 ] introduces the KEI-CQL framework, which utilizes pre-trained language models to extract semantic features from natural language queries and populate predefined slots in Cypher query templates, effectively addressing the NL2GQL challenge. The advent of large language models (LLMs) has recently revolutionized performance across various natural language processing (NLP) tasks. With their exceptional generalization and generation capabilities, LLMs have found widespread applications across diverse domains. Notably, their integration into database research has gained significant traction in recent years [ 13 , 14 , 15 , 16 , 17 ] , driven by their potential to bridge the gap between NL and structured query languages, enabling more intuitive and efficient database interactions. Recently, there has been growing research interest in applying LLMs to graph databases, with a primary focus on the NL2GQL task. This study [ 18 ] attempts to address this task through heuristic prediction and LLM-based revision. Although the method is relatively simple, it has demonstrated some effectiveness in specific domains. This work [ 8 ] leverages the interpretative strengths of smaller models during the initial ranking and rewriting stages, while capitalizing on the superior generalization and query generation capabilities of larger models for the final transformation of NL into GQL formats. This paper [ 7 ] proposes a method for constructing an NL2GQL dataset based on a domain-specific graph database and recommends using tokenization to extract the related schema and incorporate it into the context to enhance the model’s accuracy. LLM-based methods exhibit some effectiveness in solving NL2GQL tasks, but their streamlined approach carries a major challenge—error accumulation. If the related schema is extracted incorrectly, it significantly increases the likelihood of generating flawed GQL. For instance, as shown in Figure 1 , the correct related schema for the query includes the nodes “manager,” “fund,” and “stock,” along with the edges “manage” and “hold.” However, if an incorrect related schema is extracted that only includes “manager,” “fund,” and “manage,” the generated GQL might be: MATCH (f:manager {

Read the complete text