Learning-guided Prioritized Planning for Lifelong Multi-Agent Path Finding in Warehouse Automation DOI: 10.1613/jair.1.20611 Volume: 85 28 3
Han Zheng, Yining Ma, Brandon Araki, Jingkai Chen, Cathy Wu · 2026
Read Learning-guided Prioritized Planning for Lifelong Multi-Agent Path Finding in Warehouse Automation DOI: 10.1613/jair.1.20611 Volume: 85 28 3 by Han Zheng, Yining Ma, Brandon Araki, Jingkai Chen, Cathy Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-23838.txt.
Introduction
Zheng and co-authors add reinforcement learning to path planning for warehouse robots. Lifelong multi-agent path finding needs many robots to keep moving without conflicts. Classical prioritized planning is simple, but fixed priority orders adapt poorly. Their system, RL-RH-PP, learns priority orders step by step within a rolling horizon planner. An attention network outputs the orders on the fly. In warehouse simulations, it reaches the highest throughput among baselines and adapts across layouts and robot counts. It tends to favor congested robots early.
Catalog record
| CREATOR | Han Zheng, Yining Ma, Brandon Araki, Jingkai Chen, Cathy Wu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 118934 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.23838
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning-guided Prioritized Planning for Lifelong Multi-Agent Path Finding in Warehouse Automation DOI: 10.1613/jair.1.20611 Volume: 85 28 3 Han Zheng, Yining Ma, Brandon Araki, Jingkai Chen, Cathy Wu Abstract . Lifelong Multi-Agent Path Finding (MAPF) is critical for modern warehouse automation, which requires multiple robots to continuously navigate conflict-free paths to optimize the overall system throughput. However, the complexity of warehouse environments and the long-term dynamics of lifelong MAPF often demand costly adaptations to classical search-based solvers. While machine learning methods have been explored, their superiority over search-based methods remains inconclusive. In this paper, we introduce Reinforcement Learning (RL) guided Rolling Horizon Prioritized Planning (RL-RH-PP), the first framework integrating RL with search-based planning for lifelong MAPF. Specifically, we leverage classical Prioritized Planning (PP) as a backbone for its simplicity and flexibility in integrating with a learning-based priority assignment policy. By formulating dynamic priority assignment as a Partially Observable Markov Decision Process (POMDP), RL-RH-PP exploits the sequential decision-making nature of lifelong planning while delegating complex spatial-temporal interactions among agents to reinforcement learning. An attention-based neural network autoregressively decodes priority orders on-the-fly, enabling efficient sequential single-agent planning by the PP planner. Evaluations in realistic warehouse simulations show that RL-RH-PP achieves the highest total throughput among baselines and generalizes effectively across agent densities, planning horizons, and warehouse layouts . Our interpretive analyses reveal that RL-RH-PP proactively prioritizes congested agents and strategically redirects agents from congestion, easing traffic flow and boosting throughput. These findings highlight the potential of learning-guided approaches to augment traditional heuristics in modern warehouse automation. \JAIRAE Sven Koenig \JAIRTrack Multi-Agent Path Finding 1. Introduction The rapid growth of e-commerce has intensified the demand for efficient, high-throughput logistics systems ( 55 ) . Traditional manual workflows struggle with scalability and high labor costs ( 58 ) . To address these challenges, warehouse automation has transformed modern logistics by leveraging recent advances in robotics, optimization, and artificial intelligence (AI) to streamline operations and enhance efficiency ( 9 ) . Automated warehouse systems such as sortation and fulfillment centers have evolved into large-scale fleets of autonomous mobile robots (AMRs) navigating complex warehouse layouts ( 50 ) . As shown in Figure 1 , Symbotic, a leading warehouse automation company, orchestrates advanced AMRs in high-density systems, reinventing warehouse automation for increased efficiency. In such AMR systems, effective and sustained coordination is essential to alleviate congestion, reduce operational costs, and maximize throughput. Even small improvements in AMR coordination can lead to substantial efficiency gains, boosting throughput while driving down costs ( 13 ; 1 ) . A central paradigm for coordinating AMRs in warehouses is the Multi-Agent Path Finding (MAPF) problem ( 41 ) , where multiple agents navigate from start positions to assigned goals while avoiding conflicts. MAPF aims to optimize global objectives such as minimizing total travel time (flowtime) or the completion time of the last agent (makespan). Although originally developed for warehouse automation, it also has broad applications in traffic coordination ( 52 ; 57 ) , airport logistics ( 46 ) , and video games ( 11 ; 39 ) . Classic MAPF assumes a static scenario with predefined goals and performs one-shot planning. Various methods have been proposed to solve one-shot MAPF, ranging from optimal methods like Conflict-Based Search (CBS) ( 37 ) to heuristics that trade optimality for speed ( 25 ; 29 ; 32 ) . Among these, Prioritized Planning (PP) ( 12 ) is a simple yet powerful heuristic that decomposes MAPF into sequential single-agent path plans based on a predefined priority order. With a well-chosen order, PP offers impressive scalability and efficiency in large, dynamic environments. Figure 1. Robot fleet routing in a Symbotic warehouse (source: https://www.symbotic.com ). Extending beyond one-shot MAPF, real-world scenarios require a more dynamic formulation over the time dimension, known as lifelong MAPF ( 30 ) , where agents are continuously assigned new tasks after completing their current ones. This formulation is crucial for modern warehouse automation, such as fulfillment centers and sortation hubs, where robots must transition seamlessly between tasks to maximize overall throughput continuously ( 28 ) . Unlike one-shot MAPF, lifelong MAPF introduces several new challenges ( 32 ) : (i) agents repeatedly enter and leave the system, leading to continual re-coordination across time, (ii) congestion patterns evolve dynamically as tasks accumulate, requiring foresight rather than static planning, and (iii) myopic decisions can create cascading inefficiencies, as early plans directly shape future feasibility; (iv) paths with good quality need to be found within a realistic timeframe. Those challenges distinguish lifelong MAPF from one-shot MAPF and motivate the need for frameworks that explicitly model long-horizon interactions while adapting to shifting conditions. Recently, machine learning (ML) techniques have shown promise in helping address real-world complexities for one-shot MAPF ( 35 ; 18 ; 51 ) . However, they have yet to consistently outperform search-based methods in the more complex yet less explored lifelong settings ( 10 ; 40 ) . As mentioned above, the key challenge is that planning in lifelong MAPF is inherently a sequential decision-making process with causal dependencies, where each planning decision directly influences future planning outcomes. Thus, approaches that only optimize for immediate conflict avoidance or short-term efficiency-as is often sufficient in one-shot MAPF-are insufficient in the lifelong setting, since they may worsen long-term congestion or deadlock risk. Effective solutions must account for these cascading effects by maintaining a longer preview horizon to explicitly model how current decisions shape future task allocations and path interactions. This requirement complicates the direct adaptation of straightforward one-shot or short-horizon learning-based approaches, which often struggle with efficiently handling long-term dependencies and dynamic task assignment. This presents an exciting opportunity to leverage deep reinforcement learning, in novel ways that explicitly capture the causal structure of lifelong MAPF, potentially enabling learning-based methods to surpass search-based techniques in such dynamic, real-world settings. In this paper, we introduce RL-guided Rolling Horizon Prioritized Planning (RL-RH-PP) , the first hybrid framework that integrates reinforcement learning (RL) for dynamic priority order generation with search-based prioritized planning (PP). Our design choices are tailored to lifelong MAPF: the rolling horizon mechanism enables continual re-planning as new tasks arrive, while the RL-guided priority assignment explicitly captures long-horizon dependencies and adapts prioritization to evolving congestion patterns. Our motivation is to leverage reinforcement learning to address the hard-to-model sequential decision-making challenges of lifelong MAPF by capturing complex spatiotemporal interactions among agents, while integrating a simple yet efficient prioritized planning (PP) framework to achieve the best of both worlds. Importantly, PP provides an attractive backbone because it is lightweight, highly scalable, and easily extensible to dynamic settings, unlike more complex search-based solvers (e.g., CBS or PBS) whose computational cost grows rapidly with team size. The sequential structure of rolling horizon planning is naturally compatible with learning-guided optimization, as it exposes the long-horizon ordering decision as the key lever of coordination. This synergy allows us to focus the learning component on optimizing priorities, while relying on PP’s efficiency to compute collision-free paths in real time, making the overall framework achieve the best of both worlds. Specifically, RL-RH-PP casts dynamic priority assignment as a Partially Observable Markov Decision Process (POMDP) and uses a learned neural network to autoregressively decode priority orders. These orders are then fed into a