Memory Transfer Learning: How Memories are Transferred Across Domains in Coding Agents

Kangsan Kim, Minki Kang, Taeil Kim, Yanlai Yang, Mengye Ren, Sung Ju Hwang · 2026

Read Memory Transfer Learning: How Memories are Transferred Across Domains in Coding Agents by Kangsan Kim, Minki Kang, Taeil Kim, Yanlai Yang, Mengye Ren, Sung Ju Hwang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14004.txt.

Introduction

Kim and colleagues test whether coding agents can borrow memories across domains instead of staying in one task silo. A unified pool from varied domains lifts average performance by 3.7 percent, mainly by transferring meta-knowledge like validation routines rather than task-specific code. High-level insights generalize well, low-level traces often hurt through excess detail, and transfer grows with pool size and even crosses models.

Catalog record

CREATORKangsan Kim, Minki Kang, Taeil Kim, Yanlai Yang, Mengye Ren, Sung Ju Hwang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51769
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.14004

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Memory Transfer Learning: How Memories are Transferred Across Domains in Coding Agents Kangsan Kim, Minki Kang, Taeil Kim, Yanlai Yang, Mengye Ren, Sung Ju Hwang Abstract Memory-based self-evolution has emerged as a promising paradigm for coding agents. However, existing approaches typically restrict memory utilization to homogeneous task domains, failing to leverage the shared infrastructural foundations, such as runtime environments and programming languages, that exist across diverse real-world coding problems. To address this limitation, we investigate Memory Transfer Learning (MTL) by harnessing a unified memory pool from heterogeneous domains. We evaluate performance across 6 coding benchmarks using four memory representations, ranging from concrete traces to abstract insights. Our experiments demonstrate that cross-domain memory improves average performance by 3.7%, primarily by transferring meta-knowledge, such as validation routines, rather than task-specific code. Importantly, we find that abstraction dictates transferability; high-level insights generalize well, whereas low-level traces often induce negative transfer due to excessive specificity. Furthermore, we show that transfer effectiveness scales with the size of the memory pool, and memory can be transferred even between different models. Our work establishes empirical design principles for expanding memory utilization beyond single-domain silos. 1 KAIST 2 New York University 3 DeepAuto.ai https://memorytransfer.github.io/ \icml@noticeprintedtrue † † footnotetext: \dagger Equal advising. Correspondence to: [email protected] Figure 1 : Conceptual overview of Memory Transfer Learning. Unlike (A) memory-less agents or (B) single-domain self-evolving agents, (C) our approach utilizes a shared memory pool from heterogeneous coding tasks. (D) In the evaluation on diverse benchmarks, MTL outperforms a self-evolving approach. 1 Introduction As performance gains from scaling training data in language models begin to plateau, self-evolution, which leverages prior inference outcomes to enhance future performance without additional supervision, has emerged as a promising paradigm for advancing model capabilities in agents ( 9 ; 7 ) . Memory plays a central role in self-evolving agents by enabling the extraction of reusable workflows and transferable insights from past inferences and their application to subsequent tasks ( 43 ; 34 ; 24 ) . In coding agents, the memory is instantiated as code snippets, experiential knowledge including planning and debugging traces, or general programming principles ( 37 ) . Leveraging this knowledge allows agents to reference successful solution patterns in similar tasks, thereby reducing reasoning overhead, while also avoiding unnecessary failure actions in long-horizon code editing through adherence to accumulated procedural and strategic guidance, such as small-step modification heuristics and verification routines. While memory-augmented coding agents have shown promise ( 24 ) , existing approaches mostly restrict memory generation and retrieval to the same domain, typically within the same benchmark as illustrated in Figure 1 (B). However, in real-world scenarios, coding agents must handle a wide spectrum of programming problems, ranging from repository-level software engineering tasks ( 15 ) and machine learning model development ( 23 ; 27 ) to function-level competitive coding ( 14 ) . Despite this diversity, these tasks share a common underlying infrastructure, including runtime environments (e.g., Linux shells), programming languages, and cross-file dependency stacks. Current approaches that restrict memory utilization to a single domain fail to leverage this shared foundation, thereby preventing agents from exploiting a substantially richer memory pool derived from heterogeneous domains. We posit that such cross-domain memories can provide valuable guidance, often more effective than those extracted solely from the same domain, by offering transferable knowledge applicable to new problems ( Figure 1 (C)). Some prior works have explored the construction of a large unified memory pool spanning multiple task types, providing initial evidence that general reasoning experiences can support software engineering tasks ( 30 ) . However, this line of works leaves several key research questions for a practical deployment unresolved, as in Figure 1 : RQ1. Does memory from heterogeneous domains improve the performance of coding agents? RQ2. Why do transferred memories yield benefits across different domains? RQ3. Which factors in memory transfer learning most influence transfer effectiveness? To address these open questions, we conduct a systematic investigation of Memory Transfer Learning across heterogeneous domains in coding agents and derive several core findings on its mechanisms and effects. We first generate memories for each task using four different formats commonly adopted in prior works: Trajectory ( 43 ) , Workflow ( 34 ) , Summary ( 28 ) , and Insight ( 24 ) , as illustrated in Figure 2 . We then evaluate coding agent performance in zero-shot setting and under Memory Transfer Learning. The results demonstrate that Memory Transfer Learning can provide effective and transferable knowledge, improving 3.7% of average scores of 6 coding benchmarks. Our analysis yields three core findings into the mechanism of memory transfer. First, cross-domain significantly improve the performance of coding agents. Although existing self-evolving methods often overlook out-of-domain memories, our results suggest that effective memory utilization should incorporate all past experiences, including those from different domains, to enhance agent performance, as shown in Figure 1 (D). Second, the primary transferable value lies in meta-knowledge. Through qualitative analysis, we find that cross-task benefits stem not from task-specific code content but from operational know-how, such as preventing execution failures under environment constraints and task-solving routines that prioritize structural and interface inspection followed by strict validation procedures. Third, abstraction dictates transferability. By quantifying the abstraction level of each memory format, we discover a positive correlation between high-level abstraction and transfer effectiveness. Highly abstract memories, such as Insights, become task-agnostic and generalizable. In contrast, low-abstraction memories like Trajectories retain excessive task-specific details that can distract the agent, confirming that raw execution traces are less suitable for cross-task transfer. Furthermore, we provide additional insights into Memory Transfer Learning, including why negative transfer occurs, how the performance gain of MTL scales with a larger memory pool and more domains, and the potential for transferring memories across different models. In conclusion, this work presents a first holistic investigation of Memory Transfer Learning. Importantly, through extensive evaluation on six coding benchmarks, we show that existing agents’ memory usage methods, which focus on a homogeneous domain, are limited, and that there is significant room for improvement by leveraging memories from heterogeneous domains. We hope this study expands the scope of memory utilization beyond single-domain settings and stimulates further research on how to effectively leverage memory in self-evolving agents, ultimately leading to more capable coding agents. Figure 2 : Illustrative examples of four memory formats. We utilize Trajectory, Workflow, Summary, and Insight formats to analyze how different levels of information abstraction affect cross-task transferability. 2 Related Work 2.1 Coding Agents As LLM have demonstrated strong capabilities in code generation ( 26 ; 13 ; 45 ) , researchers have developed LLM-based coding agents that interact with programming environments such as bash shells ( 31 ; 32 ) through diverse systematic designs, and have evaluated them across a wide range of coding tasks. At the early stage of coding agents, they target function-level code generation tasks ( 5 ; 14 ; 35 ) in a single file. AlphaCodium ( 25 ) proposed a flow engineering in code generation which iteratively run reasoning, generation, ranking, and debugging. LDB ( 44 ) introduced a novel debugging framework with language models that leverage runtime execution information for function-level code generation. Beyond a single file level editing, CodeAgent ( 42 ) , RepoAgent

Read the complete text