Hallucination Mitigation using Agentic AI Natural Language-Based Frameworks
Diego Gosmar, Deborah A. Dahl · 2025
Read Hallucination Mitigation using Agentic AI Natural Language-Based Frameworks by Diego Gosmar, Deborah A. Dahl (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13946.txt.
Introduction
Generative AI systems state falsehoods with total confidence, and that habit corrodes trust. Diego Gosmar and Deborah A. Dahl test whether a team of AI agents can police each other. Their pipeline feeds over three hundred prompts, crafted to provoke hallucinations, into a front-end agent. Second and third level agents, using different models, then review the output. They flag unverified claims, add disclaimers, and mark speculation.
A fourth agent scores the results with new hallucination measures the authors designed. The agents talk through the OVON framework, passing structured messages that carry context and reasons. Nothing is lost between stages. The study treats hallucination less as a model flaw to patch and more as a workflow problem, one that layered review can measurably reduce.
Catalog record
| CREATOR | Diego Gosmar, Deborah A. Dahl |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-13946
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hallucination Mitigation using Agentic AI Natural Language-Based Frameworks Diego Gosmar, Deborah A. Dahl Abstract Hallucinations remain a significant challenge in current Generative AI models, undermining trust in AI systems and their reliability. This study investigates how orchestrating multiple specialized Artificial Intelligent Agents can help mitigate such hallucinations, with a focus on systems leveraging Natural Language Processing (NLP) to facilitate seamless agent interactions. To achieve this, we design a pipeline that introduces over three hundred prompts, purposefully crafted to induce hallucinations, into a front-end agent. The outputs are then systematically reviewed and refined by second- and third-level agents, each employing distinct large language models and tailored strategies to detect unverified claims, incorporate explicit disclaimers, and clarify speculative content. Additionally, we introduce a set of novel Key Performance Indicators (KPIs) specifically designed to evaluate hallucination score levels. These metrics offer a structured and quantifiable framework for assessing the impact of each agent’s refinements on the factuality and clarity of AI-generated responses. A dedicated fourth-level AI agent is employed to evaluate these KPIs, providing detailed assessments and ensuring accurate quantification of shifts in hallucination-related behaviors. A core component of this investigation is the use of the OVON (Open Voice Network) framework, which relies on universal NLP-based interfaces to transfer contextual information among agents. Through structured JSON messages, each agent communicates its assessment of the hallucination likelihood and the reasons underlying questionable content, thereby enabling the subsequent stage to refine the text without losing context. Experimental results suggest that this multi-agent, JSON-based approach not only lowers the overall hallucination scores but also renders speculative content more transparent and clearly demarcated from factual claims, improving the AI explainability level. Our findings underscore the feasibility of multi-agent orchestration and highlight the importance of maintaining a structured exchange of meta-information - particularly through formats supporting Natural Language API - to enhance the reliability and interpretability of AI-generated responses. The results demonstrate that employing multiple specialized agents capable of interoperating with each other through NLP-based agentic frameworks - such as the OVON framework - can yield promising outcomes in hallucination mitigation, ultimately bolstering trust within the AI community. K eywords Conversational AI \cdot Artificial intelligence \cdot Generative AI \cdot AI Interoperability \cdot AI Agents \cdot Multi-Agency \cdot Agentic AI \cdot Chatbot \cdot Voicebot \cdot Intelligent Assistant \cdot Multiparty AI \cdot LLM Hallucination \cdot AI Ethics \cdot Trustworthy AI \cdot Explainable AI 1 Structure of the Paper This paper is organized to systematically address the challenge of mitigating hallucinations in Large Language Models (LLMs) using Agentic AI frameworks. The first section introduces the core challenges posed by hallucinations in Generative AI LLMs and defines essential concepts, including hallucination, Agentic AI, and multi-agent orchestration. Additionally, it presents the Open Voice Network (OVON) framework [ 10 ] as a standardized approach for facilitating inter-agent communication via Natural Language-Based APIs. The experimental design and methodology are detailed in Section 3, focusing on the use of 310 carefully crafted prompts designed to induce hallucinations. This section also explains the design of the multi-agent pipeline, highlighting the specific roles of the agents and the structured use of OVON JSON messages for efficient inter-agent communication. Section 4 introduces a set of novel Key Performance Indicators (KPIs) developed to evaluate hallucination mitigation. These metrics - Factual Claim Density, Factual Grounding References, Fictional Disclaimer Frequency, and Explicit Contextualization Score - provide a structured framework for quantitatively assessing the pipeline’s effectiveness. The section also includes the formulas used to calculate the Total Hallucination Scores (THS). The empirical results are presented in Section 5, demonstrating how the multi-agent pipeline progressively reduces hallucinations. This section includes detailed visualizations to illustrate improvements in hallucination scores as prompts move through the pipeline. Section 6 follows with a practical use case, providing a step-by-step illustration of the pipeline’s application in a real-world scenario. This example offers a tangible perspective on how the system operates to mitigate hallucinations. Section 7 discusses the results in depth, analyzing variations in hallucination mitigation across different prompts and highlighting the impact of OVON-based inter-agent communication in enhancing overall system performance. The limitations of the current methodology are explored in Section 8, including its reliance on a limited set of LLMs and the need for greater transparency in their decision-making processes. The challenge of achieving full Explainable AI within these LLMs is also addressed, contrasted with the openness provided by the OVON-based agentic framework. Section 9 outlines proposals for future improvements, such as expanding the agentic framework, integrating additional specialized agents, and incorporating methodologies like Automated Design of Agentic Systems (ADAS) [ 21 ] . These advancements aim to refine the pipeline’s adaptability and effectiveness in mitigating hallucinations. Finally, Section 10 concludes the paper by summarizing the findings and emphasizing the effectiveness of multi-agent orchestration in reducing hallucinations, highlighting the potential of this approach to advance Trustworthy AI. 2 Introduction Hallucinations in the context of Large Language Models (LLMs) refer to instances where the model produces information that is factually incorrect, fabricated, or nonsensical while maintaining a confident, authoritative tone. For the remainder of this paper, we adopt the definition of “agent” as discussed by Schlosser in the Stanford Encyclopedia of Philosophy [ 19 ] . In that context, an agent is broadly understood as an entity capable of acting with intentionality and exercising a measure of control over its actions. By referencing this conceptual framework, we seek to highlight the notion of agency as it applies to our multi-agent system, wherein each agent - whether a human-guided process or an AI module - acts with a degree of autonomy and goal-directed behavior. Likewise, "Agentic AI" refers to artificial intelligence systems or architectures designed to operate as autonomous or semi-autonomous "agents" capable of performing specific tasks, making decisions, and interacting with other agents or systems in a goal-directed manner. The term "Agentic" highlights the notion of agency, where the AI acts with some level of intentionality, autonomy, and purpose within defined boundaries. The concept of Agentic AI is particularly significant in the development of modular, scalable, and transparent AI systems [ 6 ] . These systems rely on specialized agents with distinct capabilities working collaboratively to address complex problems. For instance, in the context of hallucination mitigation, one agent might focus on generating content, while others are tasked with reviewing, refining, and validating it to ensure accuracy and reliability. Similarly, in dynamic fields such as healthcare or mental health support, specialized agents can work together to manage domain-specific tasks, providing a comprehensive and cohesive solution. A recent survey [ 20 ] offers a detailed exploration of Agentic AI systems, examining their architectures, methodologies, and applications across various domains. Building on this foundation, the survey further outlines challenges and future directions for Agentic AI, including issues of explainability, interoperability, and ethical considerations. Agentic AI not only highlights the individual autonomy of these agents but also underscores the collective intelligence that emerges from their effective integration within a unified system. To explore how multi-agent collaboration can mitigate such hallucinations, we propose an empirical testing approach that leverages Natural Language–Based APIs - specifically through the OVON (Open Voice Network) interoperability standard [ 10 ] and [ 11 ] - to facilitate seamless communication among different agent layers. By injecting three hundred and