Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments

Yuxin Chen, Xiaodong Cai, Junfeng Fang, Zhuowen Han, Yu Wang, Yaorui Shi, Yi Zhang, Qi Gu, Xunliang Cai, Xiang Wang, An Zhang, Tat-Seng Chua · 2026

Read Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments by Yuxin Chen, Xiaodong Cai, Junfeng Fang, Zhuowen Han, Yu Wang, Yaorui Shi, Yi Zhang, Qi Gu, Xunliang Cai, Xiang Wang, An Zhang, Tat-Seng Chua (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-27209.txt.

Introduction

Chen and co-authors train agents under noise so they hold up in messy real settings. Their NoisyAgent frame adds two kinds of trouble: vague or shifting user input, and tool calls that fail or act oddly. Noise hits only some training runs at first, then grows harder as the model adapts. Tests show steadier behavior under noisy and shifting conditions, and the noise-trained models also gain on clean benchmarks. The authors read this as evidence that controlled exposure to flaws builds more general reasoning.

Catalog record

CREATORYuxin Chen, Xiaodong Cai, Junfeng Fang, Zhuowen Han, Yu Wang, Yaorui Shi, Yi Zhang, Qi Gu, Xunliang Cai, Xiang Wang, An Zhang, Tat-Seng Chua
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48336
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.27209

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments Yuxin Chen, Xiaodong Cai, Junfeng Fang, Zhuowen Han, Yu Wang, Yaorui Shi, Yi Zhang, Qi Gu, Xunliang Cai, Xiang Wang, An Zhang, Tat-Seng Chua Abstract Recent advances in large language models (LLMs) have facilitated the widespread deployment of LLMs as interactive agents capable of reasoning, planning, and tool use. Despite strong performance on existing benchmarks, such agents often exhibit notable degradation when deployed in real-world settings, where environments are inherently stochastic and imperfect. We argue that this discrepancy arises from a fundamental mismatch between idealized training settings and real-world interaction dynamics, where current paradigms rely on carefully curated task instructions and stable, well-controlled environments. To address this gap, we propose NoisyAgent, an agentic training framework that explicitly incorporates environmental imperfections into the agent learning process. We identify two major sources of interaction noise in real-world scenarios: user noise, which captures ambiguity and variability in user interaction, and tool noise, which reflects failures and anomalies in tool execution. We introduce such perturbations into the training pipeline by modifying user interaction patterns and simulating tool execution results within the training environment. To stabilize training while encouraging agents to handle increasingly challenging imperfection, noise is applied to only a subset of rollouts and progressively increased in difficulty as the model adapts to the current noise level. Extensive experiments demonstrate that our approach consistently improves agent robustness under noisy and dynamic environments. Our analysis reveals that training under noise condition also yields performance gains on idealized benchmarks, suggesting that controlled exposure to environmental noise promotes more generalizable reasoning and decision-making behaviors. Our findings highlight the importance of modeling interaction imperfections for bridging the gap between agent training and real-world deployment. 1 Introduction Recent advances in large language models (LLMs) have transformed them from passive text generators into interactive agents capable of reasoning, planning, and tool use [ 28 , 10 , 47 ] , enabling their widespread deployment in real-world applications. As these capabilities continue to improve [ 46 , 76 , 48 ] , LLM agents have achieved strong performance across a wide range of benchmarks [ 69 , 3 , 12 ] . However, this success does not consistently transfer to more realistic settings: when confronted with complex and dynamic environments, many agents exhibit notable performance degradation [ 5 , 84 , 67 ] . We argue that current agent learning paradigms exhibit a fundamental gap between training conditions and real-world deployment. A common characteristic shared by existing agent training paradigms is their reliance on idealized assumptions, where agents are trained with carefully curated instructions and interact with stable, well-controlled environments [ 75 , 32 , 17 ] . In contrast, real-world environments are inherently stochastic and imperfect. Users often exhibit diverse interaction styles and unpredictable behaviors [ 8 , 51 , 58 ] , while external tools may return noisy, incomplete, or even failed outputs due to various uncontrollable factors [ 54 , 65 ] . This discrepancy between training conditions and deployment environments limits the robustness of current agents, often leading to degraded performance in practical applications [ 33 , 45 , 41 ] . Figure 1 : Overview of NoisyAgent. We inject structured perturbations into both user instructions and tool responses to simulate real-world imperfections. Training is conducted via hybrid rollouts that combine clean and noisy trajectories, together with an adaptive scheduler that increases noise difficulty based on the performance gap \Delta . Policy optimization is performed with group-wise normalization to stabilize learning under heterogeneous interaction conditions. Inspired by the success of stochastic perturbations in reinforcement learning [ 50 , 34 , 82 ] , we argue that agent robustness emerges from exposure to diverse imperfections in learning process. Rather than relying on idealized training settings and expecting agents to adapt post hoc, we explicitly incorporate environmental noise and uncertainty into the agentic training process. However, how to model and introduce such noise in agentic training remains underexplored, and naively injecting noise into the training environment can easily destabilize training dynamics, making it a non-trivial challenge. Toward this goal, we propose NoisyAgent, an agentic RL method for training under noisy environments. We begin by identifying representative forms of real-world noise and developing an automated pipeline to incorporate such imperfections into the training process. Concretely, we consider two major sources of interaction noise in real-world agent scenarios: user noise, which captures ambiguity and variability in user interactions, and tool noise, which simulates execution anomalies from external tools. These perturbations are introduced by modifying user instructions and simulating tool execution results within the training environment, with perturbations applied to only a subset of rollouts for each task. Training follows a curriculum schedule. Starting from mild perturbations, we progressively increase the difficulty and ratio of noise as the model exhibits sufficient robustness at each stage. Robustness is quantified by the performance gap between idealized and perturbed environments on the same tasks. This adaptive process ensures that training remains informative rather than overwhelming, while avoiding inefficient exploration of excessively noisy regimes. Benefiting from our noise-aware training, agents achieve improved performance on benchmarks augmented with real-world noise, indicating enhanced robustness under imperfect and dynamic environments. Interestingly, we also observe consistent gains on standard, idealized benchmarks. We hypothesize that appropriately designed noise introduces controlled instability into the training environment and promotes more generalizable reasoning and decision-making. In particular, exposure to noisy and uncertain interactions encourages agents to recover from errors, resolve ambiguities, and adapt to unexpected outcomes. From this perspective, noise serves as a form of implicit difficulty augmentation, enriching the training distribution and improving robustness beyond idealized settings. Overall, our contributions can be concluded as follows: • We identify a fundamental gap between idealized agent training and real-world deployment, highlighting the importance of modeling environmental uncertainty for robust agent learning. • We develop a noise-aware training framework that systematically incorporates instruction and tool perturbations into the training environment. • Extensive experiments demonstrate that our approach consistently improves agent robustness under noisy and dynamic environments, while also yielding performance gains on standard benchmarks. 2 Preliminary 2.1 Agentic Reinforcement Learning In representative agentic training paradigm, each taks can be formalized as a Partially Observable Markov Decision Process (POMDP) [ 81 ] : \mathcal{M}=(\mathcal{S},\mathcal{A},\mathcal{O},\mathcal{T},\mathcal{R}). (1) At each step t , the agent maintains a state s_{t}=(s_{t}^{\text{env}},h_{t},q)\in\mathcal{S} , which captures the environment state s_{t}^{\text{env}} , the interaction history h_{t} , and the task prompt q . Based on the current observation o_{t}\in\mathcal{O} , the agent selects an action a_{t}\in\mathcal{A} , where the action space \mathcal{A}=\mathcal{A}_{\text{user}}\cup\mathcal{A}_{\text{tool}} includes both user interaction and tool calling invocations. Correspondingly, the observation space \mathcal{O}=\mathcal{O}_{\text{user}}\cup\mathcal{O}_{\text{tool}} consists of user-side feedback and tool execution results. Upon taking action a_{t} , the environment states evolves according to the transition function \mathcal{T}:\mathcal{S}\times\mathcal{A}\rightarrow\mathcal{S}\times\mathcal{O} , producing the next observation o_{t+1} . The training objective is to learn a policy \pi_{\theta} that maximizes the expected cumulative reward \mathbb{E}_{\tau\sim\pi_{\theta}}\!\left[\sum_{t=0}^{T}r_{t}\right] over trajectories \tau=(o_{0},a_{0},o_{1},a_{1},\ldots,o_{T}) . A widely adopted training paradigm is Reinforcement Learning with Verifiable Rewards (RLVR) [ 4 , 15 ] , where a verifier evaluates whether the final environment state s_{T}^{\text{env}} or the full trajectory \tau satisfies the task instruction given rubrics, providing a scalar reward at the trajectory level. To optimize

Read the complete text