Uncovering Systemic and Environment Errors in Autonomous Systems Using Differential Testing
Rahil P. Mehta, Yashwanthi Anand, Manish Motwani, Sandhya Saisubramanian · 2025
Read Uncovering Systemic and Environment Errors in Autonomous Systems Using Differential Testing by Rahil P. Mehta, Yashwanthi Anand, Manish Motwani, Sandhya Saisubramanian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03870.txt.
Introduction
A research team presents AIProbe in 2025, a split-test tool that sorts robot faults from world faults. When a self-run agent fails, blame may sit in its build or in a task no agent could crack. Their tool breeds fresh worlds and tasks by a spread pick of knobs.
A search plan tool, free of the agent, then tries each task on its own. Agent and plan runs side by side show which fails trace to build flaws and which to dead tasks. Across fields, the tool beats past tools on fault finds.
Catalog record
| CREATOR | Rahil P. Mehta, Yashwanthi Anand, Manish Motwani, Sandhya Saisubramanian |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-03870
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Uncovering Systemic and Environment Errors in Autonomous Systems Using Differential Testing Rahil P. Mehta, Yashwanthi Anand, Manish Motwani, Sandhya Saisubramanian Abstract When an autonomous agent behaves undesirably, including failure to complete a task, it can be difficult to determine whether the behavior is due to a systemic agent error , such as flaws in the model or policy, or an environment error , where a task is inherently infeasible under a given environment configuration, even for an ideal agent. As agents and their environments grow more complex, identifying the error source becomes increasingly difficult but critical for reliable deployment. We introduce AIProbe , a novel black-box testing technique that applies differential testing to attribute undesirable agent behaviors either to agent deficiencies, such as modeling or training flaws, or due to environmental infeasibility. AIProbe first generates diverse environmental configurations and tasks for testing the agent, by modifying configurable parameters using Latin Hypercube sampling. It then solves each generated task using a search-based planner, independent of the agent. By comparing the agent’s performance to the planner’s solution, AIProbe identifies whether failures are due to errors in the agent’s model or policy, or due to unsolvable task conditions. Our evaluation across multiple domains shows that AIProbe significantly outperforms state-of-the-art techniques in detecting both total and unique errors, thereby contributing to a reliable deployment of autonomous agents. Introduction Autonomous agents are increasingly deployed in complex real-world applications such as autonomous driving ( Yurtsever et al. 2020 ) , crop fertilization ( Gautron et al. 2022 ; Solow, Saisubramanian, and Fern 2025 ) , and elderly care ( Bardaro, Antonini, and Motta 2022 ; Mhlanga 2024 ) . Agents operating in complex settings may sometimes produce undesirable behaviors, including failure to complete the task. We refer to such behaviors as execution anomalies . Diagnosing the root cause of execution anomalies is critical for ensuring reliable, safe deployment. Execution anomalies generally arise from two broad sources: (1) agent errors : systemic errors in agent modeling or training, which results in an incorrect policy, or (2) environment errors : unfavorable environment configuration that makes task success inherently infeasible, even for an ideal agent. Agent errors may arise from model defects in the form of inaccuracies in the state representation, reward function, or both, in hand-crafted or learned model used for decision-making ( Amodei et al. 2016 ; Hadfield-Menell et al. 2017 ; Saisubramanian, Kamar, and Zilberstein 2020 ) ; or training flaws in model-free settings, such as suboptimal choices of learning algorithms or sim-to-real gaps ( Ramakrishnan et al. 2020 ) . On the other hand, some environment configurations are intrinsically unfavorable to agent success, such as poorly placed air vents in warehouses that reduce the efficiency of robot navigation ( Simon 2019 ) , and some tasks are inherently infeasible, such as painting the wall in blue color and red color at the same time. Consider a simple example: a mobile robot in a warehouse repeatedly fails to deliver packages from the counter to a storage area. Without a principled investigation, it is unclear whether the failure is due to (1) agent error : the robot’s policy is flawed because its model did not include information about avoiding slippery tiles, or its path planning algorithm may be suboptimal; or (2) environment error : a newly placed pallet may have blocked all feasible paths to the goal, making the task inherently infeasible even for an optimal agent. As both environments and agents grow more complex, identifying the source of execution anomalies becomes increasingly difficult. In practice, such anomalies are often incorrectly and reflexively attributed solely to agent errors. However, if the root cause lies in the environment configuration, no amount of training or verification will resolve the issue unless the environment itself is modified. Without a principled investigation that involves checking for alternative feasible paths using a model-independent planner or simulating task variants, it is difficult to determine whether the issue lies in the agent or the environment. While prior works have focused on testing for model errors ( He et al. 2024 ; Nayyar, Verma, and Srivastava 2022 ; Pang, Yuan, and Wang 2022 ) or using formal verification methods to provide guarantees on the occurrence of anomalies ( Corsi, Marchesini, and Farinelli 2021 ; Shea-Blymyer and Abbas 2024 ) , they do not determine whether an anomaly is due to the agent or the environment, without requiring detailed internal access to the agent. Figure 1: Overview of AIProbe We present AIProbe , a black-box technique that applies differential testing to determine whether execution anomalies are due to agent deficiencies or environment-induced infeasibility. Differential testing is a software testing methodology in which the same inputs are run through two or more independent systems (or solvers) and their outputs are compared ( McKeeman 1998 ) . If the outputs differ, it indicates error in one of those systems. AIProbe does not require access to the agent’s internal model or training data, treating the agent as a black-box system. To uncover configurations that are unfavorable for agent deployment, AIProbe systematically generates diverse environment configurations, by modifying the configurable parameters of a base environment (e.g., repositioning objects), using Latin Hypercube Sampling (LHS) ( Loh 1996 ) . For each generated configuration, AIProbe generates a range of tasks and tests whether the agent can complete them. To determine which tasks are feasible in an environment, AIProbe uses an independent search-based oracle planner that shares the agent’s action space but not its model. The planner aims to find a satisficing sequence of actions to solve these tasks. By comparing the agent’s behavior with that of the oracle planner, AIProbe determines whether the anomaly stems from the agent’s decision-making or from the infeasibility of the task itself. If the agent fails a task that AIProbe can complete safely, this suggests an agent error. If no safe plan exists, the environment itself is unsuitable for task completion. Figure 1 illustrates our approach. Note that AIProbe does not localize the modeling or training step that causes the error, nor does it reason why the task is unsolvable, and this is by design. We take the pragmatic stance that having a principled approach to perform such a high-level diagnosis is a prerequisite for a more fine-grained error localization. By comparing agent behavior with planner performance across a large suite of environment-task instances, AIProbe can automatically detect model flaws, environment-induced infeasibility, and edge-case behaviors, thereby identifying the operational boundary of safe deployment of autonomous systems. This analysis is also critical for reliability assessment ( Olamide, Kuyoro‘Shade, and Oludele 2020 ) and to generate model cards for autonomous systems ( Mitchell et al. 2019 ) . Our evaluation on five domains shows that our black-box differential testing method outperforms the state-of-the-art methods in error detection, in both discrete and continuous settings. Problem Formulation In goal-oriented sequential decision-making tasks, an agent must optimize a sequence of decisions to achieve a goal in an environment. The environment is modeled as a Markov decision process (MDP), formally defined by the tuple M=\langle S,A,T,R,s_{o},s_{G}\rangle , where S is a set of states, A is the set of all actions that an agent can take, T:S\times A\rightarrow S is the deterministic transition