On Verifiable Legal Reasoning: A Multi-Agent Framework with Formalized Knowledge Representations
Albert Sadowski, Jarosław A. Chudziak · 2025
Read On Verifiable Legal Reasoning: A Multi-Agent Framework with Formalized Knowledge Representations by Albert Sadowski, Jarosław A. Chudziak (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-00710.txt.
Introduction
Albert Sadowski and Jaroslaw A Chudziak build a multi-agent system for legal reasoning. Legal work needs precise reading of statute language. It also needs rules applied in a consistent way. AI systems that answer end to end hide their steps. That opacity makes their legal answers hard to trust or check.
Their system splits the work into clear stages. Specialized agents first extract legal concepts from statutes. They formalize the rules into checkable intermediate forms. A second stage maps case facts onto a concept schema. Symbolic inference then draws the conclusions that follow. Final answers are generated by a program, not free text. On tax calculation tasks, base models improve sharply over baseline. The gap to reasoning models narrows, with far better transparency.
Catalog record
| CREATOR | Albert Sadowski, Jarosław A. Chudziak |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-00710
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On Verifiable Legal Reasoning: A Multi-Agent Framework with Formalized Knowledge Representations Albert Sadowski, Jarosław A. Chudziak Abstract Legal reasoning requires both precise interpretation of statutory language and consistent application of complex rules, presenting significant challenges for AI systems. This paper introduces a modular multi-agent framework that decomposes legal reasoning into distinct knowledge acquisition and application stages. In the first stage, specialized agents extract legal concepts and formalize rules to create verifiable intermediate representations of statutes. The second stage applies this knowledge to specific cases through three steps: analyzing queries to map case facts onto the ontology schema, performing symbolic inference to derive logically entailed conclusions, and generating final answers using a programmatic implementation that operationalizes the ontological knowledge. This bridging of natural language understanding with symbolic reasoning provides explicit and verifiable inspection points, significantly enhancing transparency compared to end-to-end approaches. Evaluation on statutory tax calculation tasks demonstrates substantial improvements, with foundational models achieving 76.4% accuracy compared to 18.8% baseline performance, effectively narrowing the performance gap between reasoning and foundational models. These findings suggest that modular architectures with formalized knowledge representations can make sophisticated legal reasoning more accessible through computationally efficient models while enhancing consistency and explainability in AI legal reasoning, establishing a foundation for future research into more transparent, trustworthy, and effective AI systems for legal domain. K eywords Legal Reasoning \cdot Multi-Agent Systems \cdot Ontology Engineering \cdot Neural-Symbolic Integration \cdot Explainable AI 1 Introduction This paper investigates whether structured ontological representations can meaningfully improve legal reasoning performance for foundational models. To explore this question, we implement a proof-of-concept framework and conduct controlled evaluation on statutory reasoning tasks. Legal reasoning with artificial intelligence reveals a persistent performance paradox that motivates our investigation. While sophisticated reasoning models like OpenAI’s o1 can accurately analyze complex statutes, their high computational costs and processing delays make them impractical for many real-world applications. Meanwhile, efficient foundational models that could enable widespread deployment consistently fail at the logical rigor required for statutory analysis, often producing contradictory results across similar cases with no transparent reasoning pathway. Consider tax liability calculations based on federal statutes - a domain requiring both precise interpretation of legal language and systematic rule application. Recent evaluations show that foundational models achieve only 18.8% accuracy on such tasks, while reasoning models reach 87.0% accuracy but require substantially longer processing times and incur costs that limit their practical deployment. This 68.2-percentage-point gap represents a critical barrier to AI-assisted legal analysis, forcing practitioners to choose between accuracy and accessibility. Our investigation proceeds in two stages: first, we develop a working implementation of ontology-based legal reasoning system sufficient to enable systematic evaluation; second, we evaluate whether this approach provides measurable benefits over current methods. We deliberately treat the implementation as a proof-of-concept to isolate and measure the core contribution: whether structured knowledge representations can bridge the performance gap between reasoning and foundational models. We introduce Structured Ontological Legal Analysis Reasoner (SOLAR), a framework designed to test this hypothesis by decomposing statutory analysis into specialized stages with explicit inspection points. Our approach separates knowledge acquisition - where specialized agents analyze statutory text to extract legal concepts and formalize conditional rules into reusable ontological representations ( TBox ) - from knowledge application, where separate agents map case facts onto this ontological schema, apply symbolic inference, and generate final answers using a pre-constructed TBox interpreter. Evaluation on the Statutory Reasoning Assessment (SARA) dataset provides promising evidence that SOLAR can substantially narrow the performance gap between model types (Figure 3 ). Foundational models using our framework achieve 76.4% average accuracy, a notable improvement over zero-shot performance - bringing them close to reasoning models while maintaining computational efficiency advantages. These findings suggest that structured, ontology-based reasoning may offer a viable path to make sophisticated legal analysis capabilities accessible through models that offer practical deployment advantages. The framework’s modular design provides explicit inspection points throughout the reasoning process, allowing legal experts to verify extracted concepts, formalized rules, and inference steps - capabilities largely absent in end-to-end neural approaches. Our contributions include: (1) a proof-of-concept methodology for converting legal statutes into reusable ontological representations, demonstrated in the tax law domain, (2) empirical evidence suggesting that neuro-symbolic integration can meaningfully improve consistency in foundational model performance on calculation-oriented legal reasoning tasks, and (3) preliminary demonstration that structured knowledge representations may serve as an effective bridge between natural language understanding and symbolic logic in structured statutory domains with numerical outcomes. While these results are encouraging, they represent initial findings that warrant further investigation across diverse legal domains and more comprehensive evaluation frameworks. 2 Background and Related Work Our approach builds upon three interconnected research areas: formal legal ontologies for knowledge representation, large language models for automated ontology construction, and neural-symbolic integration for legal reasoning. This section reviews key developments in each area and positions our contribution within the broader landscape of AI-assisted legal analysis. 2.1 Legal Ontologies Legal ontologies provide formal, machine-processable representations of legal knowledge by organizing concepts, relationships and rules within legal domains [ 1 ] . These structures have evolved from foundational work like Valente and Breuker’s Functional Ontology of Law [ 2 ] to specialized ontologies addressing taxation, criminal law, and intellectual property [ 3 ] , reflecting the multi-layered nature of legal knowledge encompassing both abstract principles and concrete statutory provisions. Legal ontology development faces significant challenges due to legal language’s inherent ambiguity, vagueness, and open-textured concepts that resist precise formalization [ 4 ] . Engineering must navigate tensions between computational formalism requirements and legal knowledge’s conceptual richness [ 5 ] . The dynamic nature of legal systems, where interpretations evolve through legislation and judicial decisions, necessitates flexible ontological structures [ 6 ] , contributing to Felgenbaum’s "knowledge acquisition bottleneck" [ 7 ] . Development methodologies include top-down approaches starting with abstract legal concepts [ 8 ] , bottom-up methods extracting structures from legal texts [ 9 ] , and hybrid strategies [ 10 ] . Choiński et al. emphasized ontologies’ role in facilitating knowledge transfer between domain and technology experts [ 11 ] . Applications span semantic search [ 12 ] , compliance checking [ 13 ] , legal reasoning [ 14 ] , cross-jurisdictional interoperability [ 15 ] , and specialized frameworks like Legal-Onto [ 16 ] . Recent research integrates ontologies with machine learning to overcome knowledge acquisition limitations while preserving explainability [ 17 ] . 2.2 LLMs for Legal Reasoning and Ontology Engineering Large Language Models have emerged as powerful tools for both ontology construction and legal reasoning, addressing traditional challenges in knowledge acquisition and rule application. For ontology engineering, LLMs enable automatic or semi-automatic construction from diverse sources through direct prompting (zero-shot or few-shot) for term typing, taxonomic hierarchies, relationship extraction, and formal axiom generation [ 18 , 19 ] . Frameworks like LLMs4OL [ 20 ] , OntoGenix [ 21 ] , and OntoChat [ 22 ] demonstrate practical implementations, with fine-tuning significantly boosting domain-specific performance [ 18 ] . Studies show LLM-generated ontologies exhibit coherent modeling patterns resembling human-created ones [ 21 ] . Wu and Tsioutsiouliklis demonstrated that code-based knowledge graph representations (particularly Python) significantly outperform natural language and JSON for complex reasoning tasks [ 23 ] . Kommineni et al. proposed comprehensive