Towards Automated Ontology Generation from Unstructured Text: A Multi-Agent LLM Approach
Abid Talukder, Maruf Ahmed Mridul, Oshani Seneviratne · 2026
Read Towards Automated Ontology Generation from Unstructured Text: A Multi-Agent LLM Approach by Abid Talukder, Maruf Ahmed Mridul, Oshani Seneviratne (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23090.txt.
Introduction
Talukder, Mridul, and Seneviratne study auto ontology building from raw text, using domain-set insurance contracts as a set test case. A single-agent baseline reveals failures including poor design pattern fit, extra structure, and weak repeat repair. Their multi-agent setup splits building into four artifact roles: Domain Expert, Manager, Coder, and Quality Assurer. The multi-agent approach greatly improves structural quality and slightly enhances query use, driven mainly by front-loaded planning.
Catalog record
| CREATOR | Abid Talukder, Maruf Ahmed Mridul, Oshani Seneviratne |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64635 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.23090
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Automated Ontology Generation from Unstructured Text: A Multi-Agent LLM Approach Abid Talukder, Maruf Ahmed Mridul, Oshani Seneviratne Abstract Automatically generating formal ontologies from unstructured natural language remains a central challenge in knowledge engineering. While large language models (LLMs) show promise, it remains unclear which architectural design choices drive generation quality and why current approaches fail. We present a controlled experimental study using domain-specific insurance contracts to investigate these questions. We first establish a single-agent LLM baseline, identifying key failure modes such as poor Ontology Design Pattern compliance, structural redundancy, and ineffective iterative repair. We then introduce a multi-agent architecture that decomposes ontology construction into four artifact-driven roles: Domain Expert, Manager, Coder, and Quality Assurer. We evaluate performance across architectural quality (via a panel of heterogeneous LLM judges) and functional usability (via competency question driven SPARQL evaluation with complementary retrieval augmented generation based assessment). Results show that the multi-agent approach significantly improves structural quality and modestly enhances queryability, with gains driven primarily by front-loaded planning. These findings highlight planning-first, artifact-driven generation as a promising and more auditable path toward scalable automated ontology engineering. [[email protected], ] [[email protected], ] [[email protected], ] keywords Automated Ontology Generation, Multi-Agent Systems, Large Language Models (LLMs), Ontology Engineering, Knowledge Graph Construction, Competency Questions (CQs), Ontology Design Patterns (ODPs), SPARQL Evaluation, Retrieval-Augmented Generation (RAG), Artifact-Driven Workflows † † copyrightyear: 2025 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: LLMS4KGOE @ ESWC’26 † † address: Rensselaer Polytechnic Institute, Troy, New York, United States 1 Introduction Ontologies, built upon formal languages like the Web Ontology Language (OWL), provide a robust framework for representing domain knowledge in a structured, machine-readable format, enabling applications such as automated reasoning, semantic search, and data integration. However, manual ontology development is labor-intensive, error-prone, and requires significant domain expertise, which is a bottleneck that limits the scalability of knowledge-based systems. The emergence of large language models (LLMs) has opened new avenues for automating this process, but key questions remain open: how reliably can LLMs generate formally valid, semantically rich ontologies from unstructured text, and what architectural choices actually drive quality? This challenge is especially pronounced in domains where knowledge is embedded in complex natural language documents such as legal contracts, regulatory filings, technical specifications, where entities, relationships, and constraints must be inferred rather than directly read off a structured schema. Converting such unstructured text into a formal, queryable ontology demands not only language understanding but also adherence to modeling best practices that single-pass LLM generation frequently violates. We investigate these questions through a controlled experimental study. Starting from a direct LLM-based generation approach as a baseline, we identify its systematic limitations and use them to motivate a multi-agent architecture in which ontology construction is decomposed into specialized, artifact-driven roles. By holding requirement extraction and evaluation constant across both approaches, we isolate the impact of the construction strategy itself. The study surfaces concrete failure modes like redundancy, context degradation, and unreliable iterative remediation that points toward the architectural improvements needed for a more robust next-generation pipeline. 1.1 Key Contributions • An experimental study of LLM-driven ontology generation from unstructured domain text, comparing a direct generation baseline against a multi-agent architecture under controlled, identical conditions. • An automated CQ-driven evaluation framework that uses iterative, feedback-driven SPARQL query generation to measure the pragmatic usability of generated ontologies. • A diagnostic analysis of failure modes in LLM-based ontology generation, with concrete architectural insights that inform the design of future automated ontology engineering systems. 1.2 Motivating Use Case: Ontology Construction from Life Insurance Policies Life insurance policies provide a representative example of the challenges involved in transforming complex natural language documents into formal ontologies. Insurance contracts contain dense legal language describing entities (e.g., policyholder , beneficiary ), events (e.g., policy activation, reinstatement, claim submission), financial constraints (e.g., coverage limits and premium amounts), and temporal conditions (e.g., exclusion periods). These elements are often expressed implicitly across multiple clauses and require careful interpretation to model correctly. As such, insurance contracts exhibit rich relational, temporal, and constraint-heavy structures, making them a representative stress test for ontology generation. While domain-specific, the challenges (multi-entity reasoning, temporal constraints, financial modeling) generalize to other legal and technical documents. Consider a typical clause from a life insurance contract: “The beneficiary is entitled to a one-time payment of $50,000 upon the policyholder’s death.” While the statement appears straightforward in natural language, representing it formally in an ontology requires modeling multiple interacting concepts: the PaymentEvent , the Beneficiary role, the triggering DeathEvent of the policyholder, and the associated monetary value and currency constraints. Additionally, this representation must support downstream reasoning tasks, such as answering competency questions (CQs) like: What payment is the beneficiary entitled to upon the death of the policyholder? Traditional ontology engineering would require a domain expert and a knowledge engineer to manually identify these concepts, apply appropriate ontology design patterns (ODPs), and encode the resulting structure in OWL. When applied across an entire contract, often spanning dozens of pages and hundreds of clauses, this process becomes labor-intensive and difficult to scale. In this work, we use life insurance contracts as a running example to investigate how LLMs can assist in automating this process. Specifically, we examine how an LLM-driven pipeline can transform policy text into a formal ontology by first deriving CQs from contract clauses and then generating structured representations of the relevant entities, events, and constraints. Throughout the paper, examples and experiments are drawn from publicly available insurance policy documents to illustrate both the capabilities and limitations of automated ontology generation. 2 Related Work The use of LLMs to automate ontology engineering is a growing research area, as the traditional development process is complex and requires significant human expertise 19 ; 27 ; 6 ; 23 ; 5 . However, the existing work primarily focuses on (i) single-pass ontology generation, (ii) subtask-level assistance (e.g., taxonomy construction), or (iii) schema-guided population. In contrast, our work explicitly investigates construction strategy as the independent variable, introducing an artifact-driven multi-agent decomposition and isolating its impact under controlled evaluation. Lippolis et al. investigated the potential of LLMs to create OWL ontology drafts from user stories and CQs, demonstrating that new prompting techniques could outperform novice human modelers 19 . Similarly, Saeedizade et al. explored the use of LLMs for generating OWL modeling suggestions from ontological requirements, noting that advanced models could provide suggestions of sufficient quality to assist human modelers 27 . Kommineni et al. also explored a semi-automated pipeline for constructing knowledge graphs and ontologies using open-source LLMs, leveraging a judge LLM for evaluation 16 . While these studies show the promise of LLMs for initial ontology generation, a key challenge remains in ensuring the formal correctness and reliability of the output, a necessity for high-stakes domains. To address the limitations of one-shot generation, other works have focused on specific subtasks of ontology learning. Babaei et al. introduced LLMs4OL to evaluate LLMs on core tasks like term typing, concluding that fine-tuned models could assist in ontology construction 5 . Similarly, Lo et al. proposed OLLM for building the taxonomic backbone of an ontology 20 . While these advances are promising, issues such as hallucination