Ask the World Before Acting: Budgeted Environment Probing for World-Model Calibration
Xinyuan Song, Zekun Cai · 2026
Read Ask the World Before Acting: Budgeted Environment Probing for World-Model Calibration by Xinyuan Song, Zekun Cai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31422.txt.
Introduction
Song and Cai propose that a language agent should ask its environment questions before it acts. Long-horizon agents carry a private world model from step to step, and when that model drifts, a failure is locked in before the bad action happens. Probing the world can repair a belief, but each probe spends steps the task may need.
Their EnvProbe operator lets an agent ask about one belief field and write the returned answer back, under a fixed budget. Procedural beliefs, such as tool dependencies, yield to targeted checks. Spatial beliefs, such as object locations, lean on structural cues, and the agent's own confidence misleads when the world changes unseen. Experiments show mid-planning probes cut the final error in the world model when probe choices follow the task structure.
Catalog record
| CREATOR | Xinyuan Song, Zekun Cai |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.31422
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Ask the World Before Acting: Budgeted Environment Probing for World-Model Calibration Xinyuan Song, Zekun Cai Abstract Long-horizon language agents do not only choose actions; they carry a private model of the world from one decision to the next. When that model drifts, a later failure can be decided before the failing action is ever taken. We study a direct repair mechanism: before committing to the next task action, an agent may ask the environment about one belief field and write the answer back into its world model. This makes environment interaction a scarce calibration resource, not merely a way to advance the task. We introduce EnvProbe, a budgeted probing operator for structured belief tables. The useful probes are not the same everywhere. Procedural beliefs, such as tool dependencies, can often be repaired by targeted checks, but those checks spend steps that the task may need. Spatial beliefs, such as object locations and graph edges, rely more on structural cues; the agent’s own confidence can be a poor guide when the world changes off-screen. A type-stratified analysis formalizes this probe-action frontier, and controlled experiments show that mid-planning environment evidence reduces terminal world-model error when the probe policy follows the structure of the task. Code and environments are available at github.com/Hik289/Environment-reduce-error . 1 Introduction Language agents increasingly work by carrying state. They remember which tool has been initialized, where an object was last seen, which precondition is satisfied, and which edge in a graph is traversable. This running model is implicit in reasoning-and-acting agents such as ReAct, Reflexion, and LATS ( 29 ; 24 ; 32 ) , explicit in many tool-use and embodied systems ( 22 ; 20 ; 10 ; 1 ) , and central to recent web and long-horizon benchmarks ( 25 ; 27 ; 33 ; 3 ; 17 ) . The loop is simple: reason over the model, act, observe, and continue. It is also fragile, because the reasoning step quietly assumes that the model is still close enough to the environment. Long horizons make this assumption fragile. The model can drift even when every local generation looks plausible: a tool believed to be loaded may have become unavailable; a key believed to be in a room may have moved; a route that looked open may now be blocked. Once the agent plans on top of the stale premise, the eventual error looks like a bad final action, even though the failure began earlier in the belief state ( 28 ; 17 ) . This is a different failure mode from insufficient chain-of-thought or weak tool selection. The agent may have the right high-level plan but the wrong world on which to execute it. The environment itself contains the missing evidence. The agent can ask whether a particular field is still true, just as a software system can query an API, a robot can look at a drawer again, or a web agent can re-open a page before executing a dependent step. The question is not whether more information is useful in the abstract. Each check consumes the same limited horizon as task-advancing actions. An agent that probes too little acts confidently on stale beliefs; an agent that probes too much spends the episode verifying the world instead of changing it. The same tension appears in classical partially observable planning and value-of-information methods ( 12 ; 21 ; 2 ; 7 ) , but language-agent world models add a new wrinkle: the belief state is a symbolic table written by a model, and the available probe signals include noisy self-reports such as confidence and recency. Prior work shows that such confidence can be useful yet miscalibrated ( 8 ; 11 ) ; our question is when it should control an environment query. We introduce EnvProbe, a probing operator for language agents with explicit structured belief tables. At each planning step, EnvProbe scores candidate fields using task structure, belief recency, verbalized confidence, and dependency role. The chosen probe returns the current environment value for that field and updates the world model before the next plan is formed. Unlike retrieval augmentation, reflection, or asking the user for missing information ( 24 ; 9 ; 6 ; 4 ) , EnvProbe treats the environment itself as the evidence source and asks which already-populated belief should be verified now. The experiments separate two regimes. Procedural beliefs, such as tool preconditions and subgoal dependencies, benefit from targeted checks because the action trace gives clues about what may have gone stale. The same checks, however, compete with the dependency chain for scarce action slots. Spatial beliefs, such as object locations and graph edges, behave differently: structural task cues remain useful, while the agent’s own uncertainty report can mislead when exogenous changes leave little trace in the history. Our contributions are: • a budgeted environment-probing operator for structured world models in language agents; • a type-stratified mathematical account that separates belief repair from task-action displacement; • controlled environments and probe-aware metrics that expose world-model error before task success collapses; and • empirical evidence that structural probe scores reduce terminal belief error, while self-reported uncertainty must be treated as a noisy signal rather than a reliable oracle. Figure 1: Probe-action budget trade-off. Long-horizon agents can use the environment during planning to repair stale world-model fields. The benefit depends on belief type: procedural fields are easier to target but more exposed to action displacement, while spatial fields often favor structural probes over self-reported uncertainty. 2 Related Work LLM agents with implicit or explicit state. ReAct, Reflexion, Toolformer, ToolLLM, and LATS establish the now-standard pattern of interleaving language reasoning with environment or tool actions ( 29 ; 24 ; 22 ; 20 ; 32 ) . Embodied and web-agent systems further ground language plans in affordances or interactive interfaces ( 10 ; 1 ; 26 ; 33 ) . These systems update state through observations, execution traces, reflection, or memory, but they do not isolate probing as a budgeted action whose purpose is only to repair a structured belief table. Recent work on memory-environment realignment and rule-augmented memory recognizes related drift phenomena ( 30 ; 31 ) , while EnvProbe studies the selection problem: which field should be checked when only a few checks are affordable? Information gathering under partial observability. Classical POMDPs provide a formal account of belief-state planning under partial observability ( 12 ; 21 ) . Bayesian experimental design and adaptive submodularity formalize the value of information and greedy selection under uncertainty ( 2 ; 7 ; 14 ) . EnvProbe inherits the same information-action tension, but differs in two ways: the belief state is a collection of symbolic fields maintained by an LLM, and the selector uses noisy self-reports such as staleness and confidence. This makes the surrogate-quality question empirical as well as mathematical. LLM uncertainty and confidence calibration. Modern neural predictors are often miscalibrated ( 8 ) , and language models can sometimes estimate answer validity while still failing under distribution shift or open-ended generation ( 11 ) . UoT uses model-estimated uncertainty to ask informative questions ( 9 ) ; InfoSeeker plans information gathering under partial observability ( 6 ) ; ReSpAct adds speaking actions for clarification ( 4 ) .