ORCA: An Agentic Reasoning Framework for Hallucination Mitigation and Adversarial Robustness in Vision-Language Models

Chung-En (Johnny) Yu, Hsuan-Chih (Neil) Chen, Brian Jalaian, Nathaniel D. Bastian · 2025

Read ORCA: An Agentic Reasoning Framework for Hallucination Mitigation and Adversarial Robustness in Vision-Language Models by Chung-En (Johnny) Yu, Hsuan-Chih (Neil) Chen, Brian Jalaian, Nathaniel D. Bastian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-15435.txt.

Introduction

Chung-En Johnny Yu, Brian Jalaian, and Nathaniel Bastian present a guard for models that read images and text. Such models can state false facts about a photo, a flaw called hallucination. Attacks on the input can also fool them. Both flaws limit trust in real use. Their guard works at run time, with no need to retrain the model.

The guard, called ORCA, loops through four steps: observe, reason, critique, and act. It asks several small vision tools pointed questions and checks where they disagree. It keeps a record of its steps, so choices can be audited. On a known test, it lifts scores by a wide range on some models. It also resists attacks it was not trained against.

Catalog record

CREATORChung-En (Johnny) Yu, Hsuan-Chih (Neil) Chen, Brian Jalaian, Nathaniel D. Bastian
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-15435

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ORCA: An Agentic Reasoning Framework for Hallucination Mitigation and Adversarial Robustness in Vision-Language Models Chung-En (Johnny) Yu, Hsuan-Chih (Neil) Chen, Brian Jalaian, Nathaniel D. Bastian Abstract Large Vision-Language Models (LVLMs) exhibit strong multimodal capabilities but remain vulnerable to hallucinations from intrinsic errors and adversarial attacks from external exploitations, limiting their reliability in real-world applications. We present ORCA , an agentic reasoning framework that improves the factual accuracy and adversarial robustness of pretrained LVLMs through test-time structured inference reasoning with a suite of small vision models ( ≤ \leq 3B parameters). ORCA operates via an Observe–Reason–Critique–Act loop, querying multiple visual tools with evidential questions, validating cross-model inconsistencies, and refining predictions iteratively without access to model internals or retraining. ORCA also stores intermediate reasoning traces, which supports auditable decision-making. Though designed primarily to mitigate object-level hallucinations, ORCA also exhibits emergent adversarial robustness without requiring adversarial training or defense mechanisms. We evaluate ORCA across three settings: (1) clean images on hallucination benchmarks, (2) adversarially perturbed images without defense, and (3) adversarially perturbed images with defense applied. On the POPE hallucination benchmark, ORCA improves standalone LVLMs performance by +3.64% to +40.67% across different subsets. Under adversarial perturbations on POPE, ORCA achieves an average accuracy gain of +20.11% across LVLMs. When combined with defense techniques on adversarially perturbed AMBER images, ORCA further improves standalone LVLM performance, with gains ranging from +1.20% to +48.00% across evaluation metrics. These results demonstrate that ORCA offers a promising path toward building more reliable and robust multimodal systems. K eywords AI Robustness \cdot Agentic Reasoning \cdot Vision-Language Models \cdot Adversarial Attacks \cdot Hallucination 1 Introduction Figure 1: Adversarial perturbations can cause LVLMs to assert nonexistent objects injected by an attacker, and LVLMs hallucinate by their intrinsic errors, whereas ORCA corrects both. Vision-language models (VLMs) have made rapid progress in image captioning, visual question answering (VQA), and multimodal reasoning. Among them, Large Vision-Language Models (LVLMs) extend Vision-Language Pretrained (VLP) models (e.g., CLIP [ 1 ] , BLIP [ 2 ] ) with Large Language Models (LLMs), allowing them to generate semantically rich descriptions and multi-step inferences across modalities [ 3 ] . However, despite their advancements, LVLMs often hallucinate non-existent visual content due to intrinsic model errors or succumb to external adversarial image perturbations. These limitations are especially problematic in high-stakes domains such as remote sensing or medical diagnosis, where factuality and robustness are paramount [ 4 ] . Robustness in VLMs is multifaceted. First, intrinsic model errors remain widespread in state-of-the-art LVLMs, especially at the object level, where models confidently assert the existence of objects that are not present. Such hallucinations often stem from biased priors or learned co-occurrence patterns [ 3 ] . Second, the recent adversarial attacks targeting multimodal embeddings can cause models to produce confidently wrong predictions using imperceptible image or text perturbations. These attacks are often transferable across architectures, exposing a broad shared vulnerability in VLMs [ 4 , 5 ] . Importantly, hallucinations and adversarial failures are interrelated, particularly undesirable when compromised visual input is amplified by erroneous model inference [ 6 ] . Existing methods attempt to address these challenges through hallucination-aware decoding [ 7 ] , adversarial training [ 8 ] , or post-hoc correction [ 9 ] . Yet the existing methods are limited in scope, incur high training cost, require model access, or rely on static inference pipelines that cannot adapt to unexpected failures. We propose ORCA ( O bserve– R eason– C ritique– A ct), an agentic reasoning framework designed to improve the robustness of LVLMs against both hallucination and adversarial perturbations, as illustrated in Figure 1 . ORCA operates at test time through an iterative Observe–Reason–Critique–Act loop, querying the baseline LVLM and a suite of lightweight vision models to cross-validate predictions and resolve inconsistencies. Without requiring access to model weights or retraining, a ReAct-style [ 10 ] LLM-based agent dynamically generates evidential questions, triangulates answers across tools, and refines its decisions in a structured manner. While originally designed for object-level hallucination mitigation, ORCA exhibits emergent adversarial robustness through architectural diversity and inconsistency-driven reasoning. Its traceable reasoning logs provide auditable decision-making. Our key contributions are: • We propose ORCA, an agentic reasoning framework that enhances the robustness of pretrained LVLMs by reducing hallucinations and improving adversarial robustness through a test-time structured inference and correction process. ORCA integrates feedback from a suite of small vision models ( \leq 3B parameters) via an iterative Observe–Reason–Critique–Act loop. • Unlike prior approaches that rely on fine-tuning or adversarial training, ORCA operates entirely at test time without requiring access to model internals or additional data. • Empirical results demonstrate that ORCA consistently reduces hallucinations, achieving notable average gains across multiple tested models and benchmarks, and exhibits strong adversarial robustness under a range of attacks, all without requiring any adversarial training or defense mechanisms. 2 Related Work Figure 2: Overview of the ORCA framework. ORCA operates via an Observe–Reason–Critique–Act loop over a plug-in LVLM and a suite of vision tools. At each step, it generates evidential queries, gathers multi-tool responses, cross-validates inconsistencies, and refines predictions, with intermediate reasoning traces logged for auditability. Despite the impressive capabilities of LVLMs, hallucinations remain a persistent failure mode. Among various dehallucination approaches, post-hoc correction aims to reduce such hallucinations without retraining or probing the model by extending the test-time compute [ 9 ] . Post-hoc correction is particularly valuable for settings where compute resources are limited or model weights and architectures are inaccessible. Woodpecker [ 11 ] extracts object mentions from generated captions and verifies them using additional VQA and object detection models. LogicCheckGPT [ 12 ] improves output consistency by prompting the model with logically entailed variations to detect contradictions. DEHALL [ 13 ] integrates scene graphs and commonsense reasoning to check alignment between textual descriptions and image semantics in a symbolic manner. While effective in controlled scenarios, these approaches rely on static inference pipelines, require model-specific designs, and remain untested under adversarial settings. LVLMs are vulnerable to multimodal adversarial attacks due to their tightly coupled image-text embedding space, which expands the attack surface from image to multiple modalities [ 14 ] . The recent attacks like Adversarial Illusions [ 4 ] , Doubly-UAP [ 6 ] , and AttackVLM [ 5 ] reveal how imperceptible perturbations to images or prompts can trigger semantic failures in downstream inference. Moreover, these attacks are often transferable across VLMs due to similar visual encoder architectures, which broaden the range of potentially compromised VLMs [ 8 ] . While some existing defenses, such as adversarial training [ 15 ] or adversarial prompt tuning [ 16 ] , have been proposed, they typically require model access and expensive retraining. Conventional image preprocessing techniques such as JPEG compression [ 17 ] and feature squeezing [ 18 ] are widely adopted in modern attacks [ 4 , 19 ] to evaluate their attack success rate. The results often indicate that the preprocessing techniques only provide limited protection. While the recent attack, such as Double-UAP [ 6 ] , has started applying adversarial perturbations to hallucination benchmarks, existing defenses largely overlook the intersection of hallucination mitigation and adversarial robustness, particularly in cases where adversarial inputs amplify intrinsic model errors. Agentic frameworks offer an emerging alternative for enhancing

Read the complete text