Accelerating Detailed Routing Convergence through Offline Reinforcement Learning
Afsara Khan, Austin Rovinski · 2025
Read Accelerating Detailed Routing Convergence through Offline Reinforcement Learning by Afsara Khan, Austin Rovinski (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-03594.txt.
Introduction
Chip routing is one of the slowest steps in design. Afsara Khan and Austin Rovinski present a routing method in this 2025 work. Detailed routing must place wires while obeying strict, shrinking design rules.
Prior routers use iterative pathfinding for each net, which converges slowly. The authors accelerate convergence through offline reinforcement learning. Learned guidance can steer the router away from dead ends sooner. Faster routing shortens chip design cycles for ever denser processors. Offline learning reuses past routing runs, so no costly live trials are needed, and the router improves from data it already owns.
Catalog record
| CREATOR | Afsara Khan, Austin Rovinski |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-03594
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Accelerating Detailed Routing Convergence through Offline Reinforcement Learning Afsara Khan, Austin Rovinski Abstract Detailed routing remains one of the most complex and time-consuming steps in modern physical design due to the challenges posed by shrinking feature sizes and stricter design rules. Prior detailed routers achieve state-of-the-art results by leveraging iterative pathfinding algorithms to route each net. However, runtimes are a major issue in detailed routers, as converging to a solution with zero design rule violations (DRVs) can be prohibitively expensive. In this paper, we propose leveraging reinforcement learning (RL) to enable rapid convergence in detailed routing by learning from previous designs. We make the key observation that prior detailed routers statically schedule the cost weights used in their routing algorithms, meaning they do not change in response to the design or technology. By training a conservative Q-learning (CQL) model to dynamically select the routing cost weights which minimize the number of algorithm iterations, we find that our work completes the ISPD19 benchmarks with 1.56 × \times average and up to 3.01 × \times faster runtime than the baseline router while maintaining or improving the DRV count in all cases. We also find that this learning shows signs of generalization across technologies, meaning that learning designs in one technology can translate to improved outcomes in other technologies. I Introduction Detailed routing remains one of the most challenging aspects of modern physical design due to the increasing complexity of scaling feature sizes. As technology nodes advance, the number of design rules increases dramatically and leads to significantly increased runtime for detailed routing. Prior work in detailed routing has leveraged techniques such as Lee’s Algorithm [ 1 ] , A* search [ 2 ] , line search [ 3 ] , and others to perform detailed routing. Most notably, Dr. CU [ 4 ] and OpenROAD [ 5 , 6 ] are the leading open-source detailed routers capable of solving the ISPD18 and ISPD19 detailed routing benchmarks. In both cases, the general routing algorithms are the same: 1) A pathfinding algorithm such as A* or Dijkstra’s algorithm is used to find the minimum cost path for each routed net, 2) a design rule check (DRC) is run to evaluate any DRVs, 3) violation costs are added to the corresponding nodes on the grid, and 4) the route is ripped up and rerouted to avoid violations. An “iteration” is defined as the number of times the algorithm must ripup and reroute nets to achieve a DRV-free solution, which can become prohibitively expensive for high-density designs. Our major insight is that prior work only performs static costing of search weights, that is, the costs that are applied to violations do not change in response to the design. In the case of Dr. CU, the violation costs are static during the entire algorithm. For OpenROAD, the costs are adjusted based solely on the current ripup iteration number. One of the great challenges in trying to improve over the baseline implementations is that altering the violation costing can frequently lead the detailed router towards worse solutions – solutions which take longer to converge to 0 DRVs or do not converge 0 DRVs. During our study, we find that random exploration of weights leads to a significantly higher number of worse solutions than better solutions, and weights which work well on one design may not translate to another design. Therefore, the challenge in improving weight selection relies on both 1) finding weights which enable the router to converge to 0 DRVs and 2) converging to 0 DRVs in fewer iterations than the baseline approach. In this work, we propose using reinforcement learning (RL) to learn from prior design implementations and determine sequences of weights which will converge to 0 DRVs in the fewest number of iterations. We use perturbation data sampling [ 7 , 8 ] to explore 350-400 weight sequences per design in our training set, and then we train a conservative Q-learning (CQL) model [ 9 ] on these samples to predict the best weights for the next iteration in order to minimize iterations to convergence. We find that on the ISPD19 benchmark, our work converges in 5% fewer iterations on average and up to 31%. This translates to a runtime speedup of 1.56 \times on average and up to 3.01 \times . One of our key results is that we find that not only are the current weight schedules suboptimal, but that the conventional wisdom for weight selection is dramatically different from the weights that our RL model selects for improved results. We make the following contributions in this work: • Development of a detailed routing training dataset using perturbation sampling and random exploration techniques • Identification of critical state variables and reward functions suitable for RL models in detailed routing • Implementation and evaluation of an offline RL approach using the conservative Q-learning (CQL) model to predict optimal cost weights for detailed routing. • Demonstration of robustness across diverse designs • Analysis of cost weights vs. routing performance to demonstrate that learned weights can achieve better performance by disregarding conventional practice. • We aim to open-source all source code for training and inference along with publication of this paper. The rest of the paper is organized as follows. Section II discusses related work, Section III discusses our methodology for data generation, Section IV discusses the model architecture and training, Section V discusses the experimental results, and Section VI provides our conclusions. II Related Work II-A Detailed Routers The foundation of modern detailed routing traces back to Lee’s maze routing algorithm [ 1 ] , a breadth-first search (BFS) that guaranteed minimum cost paths. Rapid notable improvements on this base algorithm included the A* search using heuristics to guide the search [ 2 , 10 ] , along with techniques like line search [ 3 ] to speed up execution. Core strategies in modern routers also include the iterative rip-up and reroute and sometimes leverage multicommodity flow concepts [ 11 ] . TritonRoute [ 12 ] utilizes the prior findings to employ an iterative A*-based search with partitions, enhanced by dynamic boundary adjustments. Similarly, Dr. CU [ 4 ] uses Djiksta’s algorithm coupled with a sparse grid-graph and partitioning to efficiently route nets. As newer technology nodes arrive, routing research has shifted focus towards techniques such as gridless pin access [ 13 ] , routing under complex patterning like SADP [ 14 , 15 ] , minimum area-sensitive path search [ 16 ] , and ILP-based formulations [ 11 ] . In terms of academic routers, OpenROAD [ 6 ] provides state-of-the-art performance, achieving 0 DRVs on the ISPD18 [ 17 ] benchmark and 0 DRVs on all but one test case on ISPD19 [ 18 ] . OpenROAD’s router is derived from TritonRoute [ 19 , 12 ] , but several enchancements have been made and it is actively maintained to continue improving detailed routing results. In this work, we propose using dynamic weighting of costs which are learned from prior designs. Prior works use a static scheduling of weights which do not depend on the design. Additionally, we find that RL-generated weights tend to differ significantly from the expert-curated weights in the baseline