LUMINA: Long-horizon Understanding for Multi-turn Interactive Agents
Amin Rakhsha Thomas Hehn Pietro Mazzaglia, Fabio Valerio Massoli Arash Behboodi Tribhuvanesh Orekondy · 2026
Read LUMINA: Long-horizon Understanding for Multi-turn Interactive Agents by Amin Rakhsha Thomas Hehn Pietro Mazzaglia, Fabio Valerio Massoli Arash Behboodi Tribhuvanesh Orekondy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16649.txt.
Introduction
Amin Rakhsha and a team at Qualcomm AI Research study AI agents that act over long spans. Large language models do well on short, single tasks. They still fail on jobs that run through many turns. Such jobs need plans, memory of state, and long text in mind. The team's 2026 preprint probes why these jobs stay hard.
The study asks which skill counts most for success. It weighs the worth of plans, state tracking, and long context use. The aim is to learn where gains will help most. That guide can shape how labs train the next agents. It also helps buyers judge what agents can truly do.
Catalog record
| CREATOR | Amin Rakhsha Thomas Hehn Pietro Mazzaglia, Fabio Valerio Massoli Arash Behboodi Tribhuvanesh Orekondy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16649
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LUMINA: Long-horizon Understanding for Multi-turn Interactive Agents Thanks: Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc. Amin Rakhsha Thomas Hehn Pietro Mazzaglia, Fabio Valerio Massoli Arash Behboodi Tribhuvanesh Orekondy Abstract Large language models can perform well on many isolated tasks, yet they continue to struggle on multi - turn, long - horizon agentic problems that require skills such as planning, state tracking, and long context processing. In this work, we aim to better understand the relative importance of advancing these underlying capabilities for success on such tasks. We develop an oracle counterfactual framework for multi-turn problems that asks: how would an agent perform if it could leverage an oracle to perfectly perform a specific task? The change in the agent’s performance due to this oracle assistance allows us to measure the criticality of such oracle skill in the future advancement of AI agents. We introduce a suite of procedurally generated, game - like tasks with tunable complexity. These controlled environments allow us to provide precise oracle interventions, such as perfect planning or flawless state tracking, and make it possible to isolate the contribution of each oracle without confounding effects present in real - world benchmarks. Our results show that while some interventions (e.g., planning) consistently improve performance across settings, the usefulness of other skills is dependent on the properties of the environment and language model. Our work sheds light on the challenges of multi - turn agentic environments to guide the future efforts in the development of AI agents and language models. 1 Introduction Large Language Models (LLMs) have demonstrated exceptional performance across a wide range of tasks, including natural conversations ( 27 ; 2 ) , question answering ( 34 ) , competitive coding ( 3 ; 32 ) , and mathematical reasoning ( 6 ; 10 ) . Consequently, given their general-purpose capabilities, a natural question that is actively being explored is whether language models can be leveraged as multi-turn agents, i.e., whether they can iteratively perceive, reason, and take strategic actions towards achieving a distant goal. Such tasks introduce a host of new challenges, such as maintaining coherence over multiple turns and long contexts, reasoning over multiple dynamic pathways, recovering from errors, identifying the right tools for the task, and efficiently tracking state and progress without explicit feedback. Numerous benchmarks quantitatively analyze multi-turn agent capabilities across various domains, ranging from function calling ( 19 ) , web navigation ( 15 ) , interactive coding ( 28 ) , human interaction ( 18 ) , and game-playing ( 9 ) . Ongoing results suggest that there is plenty of progress to be made, e.g., on AppWorld ( 28 ) , GPT-4o has a success rate of 30.2% and open-weight models such as LLama3-70B ( 8 ) achieve 7.0%. As many of the benchmark analyses report, pushing progress requires enabling numerous skills, such as efficient task decomposition, planning, state tracking, and information gathering. Even though the necessity of these skills has been individually established in prior work, their relative importance on the agent’s success remains an open question. In this paper, our goal is to understand which of these skills (or combination thereof) are the bottleneck to make progress towards capable multi-turn agents. To assess skill importance, we introduce an oracle intervention framework that poses counterfactual questions: What if the agent had access to the information that skill provides? Using this framework, we investigate three oracle interventions and their combinations: planning , which provides a description of a single-turn subtask on the optimal trajectory, state tracking , which provides an accurate state of the agent, and history pruning , which rewrites the context to remove distractors and provide a compact summary. Measuring the impact of individual skills on an agent’s success in multi-turn, goal-oriented tasks is challenging. Real-world ablations are cumbersome and often impractical because these tasks may admit numerous equally optimal paths to the goal, making exhaustive oracle annotations intractable. To overcome this, we introduce procedurally-generated game environments where optimal actions and strategies can be computed at any step, enabling systematic oracle interventions. We consider three game-playing multi-turn tasks: ListWorld to cautiously mutate a python list using only pop actions, TreeWorld to find a particular node in a tree iteratively using only get_children actions, and GridWorld to spatially navigate a 2d grid to reach a target location while avoiding holes. All environments are configurable, and importantly, enable us to inject accurate oracle information at any point of the agent’s trajectory. Our framework enables us to examine the capabilities of multi-turn agents along multiple dimensions (e.g., task complexity, model size, influence of specific oracle interventions) and helps us providing insights into L ong-horizon U nderstanding of M ulti-turn IN teractive A gents ( LUMINA ). First, we observe a significant discrepancy with the low success rates over long-horizon trajectories in spite of high accuracy per step (i.e., whether the chosen action is from the set of optimal actions), indicating that compounding errors represent a major challenge ( 24 ; 17 ) . Second, we find that oracle interventions generally improve the success rate, but their effectiveness varies depending on model size. Finally, different environments have distinct characteristics that translate to varying gains from the interventions and measurable performance discrepancies between success rate and step accuracy. Overall, our findings present a double-edged picture: while improving specific skills (enabled by oracles interventions in our case) generally help the LLM-based agents solving multi-turn tasks, fully bridging the gap likely requires exploiting environment and model-specific understanding. 2 Related Works LLMs and Agents. Agent-based systems, defined by a policy interacting with an environment to achieve goals and receive rewards ( 23 ) , have a long history. Recent work shows that language models can serve as the policy ( 12 ; 36 ) , the environment as world model ( 11 ) , or the reward function ( 38 ; 37 ) . We focus on LLMs as policies that auto-regressively sample next actions. A prominent example is ReAct prompting ( 36 ) , which interleaves reasoning and task-specific actions. ReAct has proven effective across domains, from game playing ( 30 ) to interactive coding ( 28 ) . In this work, we adopt ReAct-based prompting to elicit dynamic reasoning and planning from LLMs. LLM Agent Capabilities. The are several essential skills for agents beyond language understanding and reasoning. Agents must execute admissible actions (e.g., tools, function calls) with correct arguments ( 22 ; 20 ) . As there may be multiple paths towards the goal, agents should support multi-path reasoning ( 4 ; 35 ) and dynamic re-planning ( 25 ) to adapt to feedback. At each turn, effective decision-making can require both short-term (context window) and long-term memory (external storage) ( 25 ; 13 ; 31 ) . Additionally, agents might need to track evolving hidden environment states influenced by prior actions ( 7 ; 29 ) . These, and potentially more, capabilities collectively underpin robust agentic behavior. Characterizing Bottlenecks for Agents. Approaches to assessing model effectiveness in agentic tasks include holistic benchmarks ( 28 ; 20 ) and capability-level analysis. Benchmarks show a clear trend: larger models outperform smaller ones, prompting the question—what drives these gaps? Few works exist