Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents

Yiming Du 1 , Baojun Wang 2 , Yifan Xiang 1 , Zhaowei Wang 3 , Wenyu Huang 4 , Boyang Xue 1 , Bin Liang 1 , Xingshan Zeng 2 , Fei Mi 2 , Haoli Bai 2 , Lifeng Shang 2 , Jeff Z. Pan 4 , Yuxin Jiang 2 , Kam-Fai Wong 1 1 The Chinese University of Hong Kong 2 Huawei Technologies Co.,Ltd 3 HKUST 4 The Uni · 2025

Read Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents by Yiming Du 1 , Baojun Wang 2 , Yifan Xiang 1 , Zhaowei Wang 3 , Wenyu Huang 4 , Boyang Xue 1 , Bin Liang 1 , Xingshan Zeng 2 , Fei Mi 2 , Haoli Bai 2 , Lifeng Shang 2 , Jeff Z. Pan 4 , Yuxin Jiang 2 , Kam-Fai Wong 1 1 The Chinese University of Hong Kong 2 Huawei Technologies Co.,Ltd 3 HKUST 4 The Uni (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-20092.txt.

Introduction

Yiming Du leads a thirteen-author team teaching chatbots to remember time properly. Across long, multi-session talks, today's long-context models lose track. They miss which past sessions bear on a when question. Noise piles up, and reasoning fails. Memory-T1 learns what to keep, with reinforcement learning. It sweeps the talk history in a coarse-to-fine pass.

Filters on time and relevance first prune the history to candidates. A learned selector then picks the exact evidence sessions. Its rewards cover answer accuracy, grounding in evidence, and time consistency at two levels. On the Time-Dialog benchmark, Memory-T1 set a new open-source best with a seven-billion-parameter model. It beat a fourteen-billion baseline by a clear margin. It stayed sound at one hundred twenty-eight thousand tokens, where baselines collapsed.

Catalog record

CREATORYiming Du 1 , Baojun Wang 2 , Yifan Xiang 1 , Zhaowei Wang 3 , Wenyu Huang 4 , Boyang Xue 1 , Bin Liang 1 , Xingshan Zeng 2 , Fei Mi 2 , Haoli Bai 2 , Lifeng Shang 2 , Jeff Z. Pan 4 , Yuxin Jiang 2 , Kam-Fai Wong 1 1 The Chinese University of Hong Kong 2 Huawei Technologies Co.,Ltd 3 HKUST 4 The Uni
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-20092

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents Yiming Du 1 , Baojun Wang 2 , Yifan Xiang 1 , Zhaowei Wang 3 , Wenyu Huang 4 , Boyang Xue 1 , Bin Liang 1 , Xingshan Zeng 2 , Fei Mi 2 , Haoli Bai 2 , Lifeng Shang 2 , Jeff Z. Pan 4 , Yuxin Jiang 2 , Kam-Fai Wong 1 1 The Chinese University of Hong Kong 2 Huawei Technologies Co.,Ltd 3 HKUST 4 The University of Edinburgh {ydu, kfwong}@se.cuhk.edu.hk, [email protected] Abstract Temporal reasoning over long, multi-session dialogues is a critical capability for conversational agents. However, existing works and our pilot study have shown that as dialogue histories grow in length and accumulate noise, current long-context models struggle to accurately identify temporally pertinent information, significantly impairing reasoning performance. To address this, we introduce Memory-T1 , a framework that learns a time-aware memory selection policy using reinforcement learning (RL). It employs a coarse-to-fine strategy, first pruning the dialogue history into a candidate set using temporal and relevance filters, followed by an RL agent that selects the precise evidence sessions. The RL training is guided by a multi-level reward function optimizing (i) answer accuracy , (ii) evidence grounding , and (iii) temporal consistency . In particular, the temporal consistency reward provides a dense signal by evaluating alignment with the query time scope at both the session-level (chronological proximity) and the utterance-level (chronological fidelity), enabling the agent to resolve subtle chronological ambiguities. On the Time-Dialog benchmark, Memory-T1 boosts a 7B model to an overall score of 67.0%, establishing a new state-of-the-art performance for open-source models and outperforming a 14B baseline by 10.2%. Ablation studies show temporal consistency and evidence grounding rewards jointly contribute to a 15.0% performance gain. Moreover, Memory-T1 maintains robustness up to 128k tokens, where baseline models collapse, proving effectiveness against noise in extensive dialogue histories. The code and datasets are publicly available at https://github.com/Elvin-Yiming-Du/Memory-T1/ 1 Introduction Recent advances in memory architectures and large language models (LLMs) have substantially improved the capabilities of conversational agents ( Yu et al. 2025 ; Zhong et al. 2024 ; Xu et al. 2025 ) . Increasingly, these agents are expected to support long-term multi-session interactions ( Du et al. 2025b ; Ge et al. 2025 ) , where a central challenge is understanding and reasoning about temporal relationships across dialogue histories ( Wu et al. 2025 ; Maharana et al. 2024 ) . Without this capability, agents may incorrectly order past events, conflate information from different sessions, and ultimately generate inconsistent or inaccurate answers. For example, as shown in Figure 1 , correctly resolving a query such as “What time did Emi mention that some ‘Suits’ characters were together at the Golden Globes?” requires the agent to locate the relevant mention in the dialogue history, understand the key relative temporal expression (“last night”), and grounding it to the correct session date (“10.01.2024”) to infer the accurate date “January 9, 2024”. Ultimately, temporal reasoning is essential for factual consistency in long, noisy conversations. However, existing approaches ( Yu et al. 2025 ; Xu et al. 2025 ) remain inadequate for temporal reasoning in conversation. General-purpose long-context models ( Team 2024 ; Guo et al. 2025b ; Wang et al. 2025 ) treat dialogue history as flat text and fail to locate or resolve temporal expressions, leading to steep performance degradation on noisy, extensive conversations ( Wu et al. 2025 ; Maharana et al. 2024 ) . Time-aware frameworks such as TReMu ( Ge et al. 2025 ) handle explicit expressions but struggle with ambiguous ones like “the week before that”, and error accumulation from inferred event summaries undermines robustness. Reinforcement learning (RL) approaches such as Time-R1 ( Liu et al. 2025 ) rely heavily on structured metadata, making them ineffective for unstructured multi-session dialogues. Thus, a robust, scalable solution for temporal reasoning in dialogue remains an open challenge. To bridge this gap, we introduce Memory-T1 , a RL-based memory retrieval framework designed for temporal reasoning that combines coarse-to-fine retrieval strategy with a multi-level reward design. In the Candidate Generation phase, the query temporal scope is predicted using an LLM to prune the dialogue history search space, which acts as a hard filter to prune irrelevant sessions. This is followed by a relevance-based retriever to produce a small, high-recall candidate set of sessions. This phase efficiently narrows the vast memory pool to a manageable context, setting the stage for a more precise analysis. In the fine-grained selection phase, an RL agent identifies the precise evidence sessions. Training such an agent is challenging because answer-only supervision provides very sparse signals. To overcome this, we design a dense, multi-level reward function. Beyond answer accuracy ( R_{a} ) and evidence grounding ( R_{g} ), we introduce a novel temporal consistency reward ( R_{t} ) that explicitly evaluates (1) session-level chronological proximity and (2) utterance-level temporal density. By rewarding temporally coherent and contextually concentrated evidence, this structured signal provides richer supervision, enabling the agent to resolve ambiguous temporal expressions and to generalize more robustly to noisy, long-context dialogues. We validate Memory-T1 on the Time-Dialog ( Wei et al. 2025 ) and LoCoMo ( Maharana et al. 2024 ) benchmarks. Results show that Memory-T1 achieves state-of-the-art temporal reasoning performance, substantially improving robustness on contexts up to 128k tokens. Notably, Memory-T1 enables a 7B model to outperform a 14B baseline, highlighting the effectiveness of temporal-aware retrieval and dense reward optimization. The key contributions are: (1) A coarse-to-fine memory retrieval framework that efficiently narrows dialogue histories into high-quality candidates before fine-grained evidence selection. (2) A novel dense reward design for RL-based retrieval introducing temporal consistency signals at both session and utterance levels, providing insights into training robust temporal-aware retrieval models by overcoming sparse reward limitations. (3) State-of-the-art performance, with Memory-T1 achieving top results and maintaining accuracy under extremely long and noisy conversational contexts. Figure 1: Multi-session QA with time-event annotations. Time range marks when an event or query occurs, either a duration or an instantaneous point (start and end coincide). Event span highlights key evidence in the utterance. 2 Related Work Temporal Reasoning in LLMs. Temporal reasoning has become an active area of research for LLMs ( Song et al. 2025 ; Wei et al. 2025 ; Liu et al. 2025 ) . Benchmarks such as TimeBench ( Chu et al. 2024 ) and TIME ( Wei et al. 2025 ) reveal that even strong models struggle with temporal relationships, event ordering, factual consistency, and long-range reasoning. To fill these gaps, prior work has aligned knowledge with temporal contexts ( Zhao et al. 2024 ) , introduced specialized training such as Timo ( Su et al. 2024 ) and TG-LLM ( Xiong et al. 2024 ) , or applied RL, as in DeepSeek-R1 ( Guo et al. 2025b ) and Time-R1 ( Liu et al. 2025 ) . However, these methods often depend on explicit supervision or handcrafted structures, limiting their applicability to multi-session dialogue. Furthermore, temporal reasoning has recently become an important problem in the memory of LLMs. TReMu ( Ge et al. 2025 ) leverages memory via timeline summaries but relies on timestamp accuracy for temporal reasoning. A few memory-related works ( Mai et al. 2025

Read the complete text