CodePDE: An Inference Framework for LLM-driven PDE Solver Generation

Shanda Li [email protected], Tanya Marwah [email protected], Junhong Shen [email protected], Weiwei Sun [email protected], Andrej Risteski [email protected], Yiming Yang [email protected], Ameet Talwalkar [email protected] · 2025

Read CodePDE: An Inference Framework for LLM-driven PDE Solver Generation by Shanda Li [email protected], Tanya Marwah [email protected], Junhong Shen [email protected], Weiwei Sun [email protected], Andrej Risteski [email protected], Yiming Yang [email protected], Ameet Talwalkar [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08783.txt.

Introduction

Shanda Li, Tanya Marwah, Junhong Shen, Weiwei Sun, Andrej Risteski, Yiming Yang, and Ameet Talwalkar use language models to write equation solvers in a 2025 paper. Partial differential equations model most physical systems in science. Classic solvers need expert skill and heavy compute. Neural solvers need large training sets and can be opaque.

Their frame, CodePDE, treats solver writing as a code task. It tests model skill in reasoning, debugging, and self-repair. With strong test-time methods, models did well on key equation types. The team found trade-offs between solver safety and flair. They also map where models fail on hard tasks. The work guides builders of AI science tools.

Catalog record

CREATORShanda Li [email protected], Tanya Marwah [email protected], Junhong Shen [email protected], Weiwei Sun [email protected], Andrej Risteski [email protected], Yiming Yang [email protected], Ameet Talwalkar [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE98 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-08783

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CodePDE: An Inference Framework for LLM-driven PDE Solver Generation Shanda Li [email protected], Tanya Marwah [email protected], Junhong Shen [email protected], Weiwei Sun [email protected], Andrej Risteski [email protected], Yiming Yang [email protected], Ameet Talwalkar [email protected] Abstract Partial differential equations (PDEs) are fundamental to modeling physical systems, yet solving them remains a complex challenge. Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive, while neural-network-based solvers require large training datasets and often lack interpretability. In this work, we frame PDE solving as a code generation task and introduce CodePDE, the first inference framework for generating PDE solvers using large language models (LLMs). With CodePDE, we present a thorough evaluation on critical capacities of LLM for PDE solving: reasoning, debugging, self-refinement, and test-time scaling. CodePDE shows that, with advanced inference-time algorithms and scaling strategies, LLMs can achieve strong performance across a range of representative PDE problems. We also identify novel insights into LLM-driven solver generation, such as trade-offs between solver reliability and sophistication, design principles for LLM-powered PDE solving agents, and failure modes for LLM on hard tasks. These insights offer guidance for building more capable and reliable LLM-based scientific engines. Reviewed on OpenReview: https://openreview.net/forum?id=eG3Qy5Oux6 \tikz [baseline=(n.base)] \node [ rounded corners=4pt, fill=black!5, draw=black!5, inner sep=6pt, font= ] (n) Github; \tikz [baseline=(n.base)] \node [ rounded corners=4pt, fill=black!5, draw=black!5, inner sep=6pt, font= ] (n) HuggingFace; 1 Introduction Partial differential equations (PDEs) are foundational for modeling complex phenomena in science and engineering, yet developing effective numerical solvers remains a substantial challenge. Traditional numerical methods demand significant computational resources and deep domain expertise for implementation. Crafting robust and efficient solvers typically involves meticulous manual tuning, extensive debugging, and specialized numerical analysis knowledge. The success of deep learning has motivated the development of neural PDE solvers ( Raissi et al., 2019 ; Lu et al., 2021 ; Li et al., 2022b , e.g.,) and multiphysics foundation models ( McCabe et al., 2024 ; Hao et al., 2024 ; Shen et al., 2024b ; Herde et al., 2024 , e.g.,) to automate PDE solving. However, training these models requires large amounts of data, and their outputs lack transparency, making it difficult to understand how and why a particular solution is produced. Meanwhile, large language models (LLMs) have been increasingly applied to complex mathematical and scientific challenges ( Romera-Paredes et al., 2024 ; Shen et al., 2024a ; Shen et al., 2025 ; Chan et al., 2025 ; Sun et al., 2025 ; Zhou et al., 2025 ; Li et al., 2026 , e.g.,) . Central to these advancements is the observation that code acts as a versatile intermediary between natural language and structured symbolic scientific computations. Inspired by this insight, a compelling alternative paradigm for PDE solving emerges: automated generation of executable solver code directly from high-level natural language descriptions of PDEs. This emerging paradigm raises a critical and open research question: Can LLMs generate effective and efficient solver code for PDEs? Despite the conceptual simplicity, naively applying LLMs to PDE solving may underutilize their capabilities, neglecting recent progress in inference-time algorithms ( Welleck et al., 2024 ) and scaling strategies ( Snell et al., 2025 ) . We introduce CodePDE, an inference framework for LLM-driven automated PDE solver generation. Given a problem description, CodePDE instructs LLMs to produce diverse solver implementations (e.g., finite difference, spectral methods, or other novel approaches). CodePDE further incorporates mechanisms for code repair, iterative refinement, test-time scaling, and rigorous evaluation of accuracy, efficiency, and convergence, as depicted in Figure 1 . Designed with forward compatibility and modularity, CodePDE supports both local deployments and API-based interfaces, and readily integrates with a variety of inference strategies and agentic workflows. Deploying CodePDE with 16 strong LLMs, we conduct systematic evaluations across four core capabilities for PDE solver generation: • Reasoning: Generating numerically sound solvers through chain-of-thought. LLMs can explore the combination of a wide range of methods, such as finite difference, finite volume, and spectral methods for spatial domains, as well as Explicit Euler, Runge-Kutta, and IMEX for time integration, yielding diverse solver strategies. • Debugging: Identifying and autonomously correcting code errors based on runtime feedback. The average rate of bug-free solver generation increases from 41% to 84% after applying iterative self-debugging, demonstrating the effectiveness of automated error repair. • Refinement: Improving solver accuracy through feedback-based improvement. We find that enabling self-refinement consistently enhances solution quality across all tested PDEs, underscoring the value of feedback-driven optimization. • Test-Time Scaling: Boosting solution quality by scaling up inference compute. Using best-of- n sampling, we observe that solution accuracy improves with increased inference budget (Figure 3 ), highlighting a practical scaling law for LLM-generated solvers. These capabilities constitute essential building blocks for LLM-powered scientific computing. Our experiments demonstrate that LLMs, when paired with inference-time techniques–such as automated debugging ( Chen et al., 2024 ) , self-refinement ( Madaan et al., 2023 ) , and test-time scaling ( Snell et al., 2025 ) –achieve performance comparable to tailored numerical solvers and specialized PDE solving softwares. Further ablation study shows that these techniques are critical ingredients for building highly accurate PDE-solving agentic workflows. CodePDE also provides a platform for systematically evaluating LLMs on solver generation, uncovering several key insights. For example, our analysis identifies a trade-off between solver reliability and sophistication, where some models conservatively produce simple, robust solvers while others explore more diverse complex methods. In addition, we find that code generation and refinement may be distinct skills for LLMs: while advanced reasoning models like o3 and DeepSeek-R1 excel at generating high-quality solvers from scratch, they do not consistently outperform standard models like GPT-4o and DeepSeek-V3 in the refinement stage. In summary, our contributions are threefold: (1) We frame PDE solving as a code generation task and develop CodePDE, the first inference framework to combine PDE domain knowledge with the self-improvement and test-time scaling capabilities of LLMs. (2) We show that CodePDE unlocks the capabilities of LLM to generate high-quality PDE solvers. (3) We present the first comprehensive study of LLM-generated PDE solvers, analyzing their accuracy, efficiency, numerical scheme diversity, and failure modes–offering new insights into the potential and limitations of LLMs in scientific computing. 2 Related Works: PDE Solving and the Emerging Role of LLMs Numerical Methods. Traditional numerical solvers approximate PDE solutions via domain discretization. Finite difference method (FDM) ( LeVeque, 2007 ) uses grid-based differences to estimate derivatives; finite element method (FEM) ( Zienkiewicz and Taylor, 2013 ) approximates solutions over mesh elements; and spectral methods ( Canuto et al., 2007 ) represent solutions as sums of global basis functions. These solvers offer convergence guarantees and error bounds, but they require expert knowledge to implement and are computationally intensive, particularly for high-dimensional problems. In recent years, a few softwares have emerged to streamline the use of these numerical methods ( Zwicker, 2020 ; Burns et al., 2020 ; Scroggs et al., 2022 ) . Neural PDE Solvers. Neural network methods such as PINNs ( Raissi et al., 2019 ; Wang et al., 2022 ; He et al., 2023 ) and neural operators ( Lu et al., 2021 ; Li et al., 2022b ; Li et al., 2025 ) provide data-driven alternatives. While bypassing some traditional bottlenecks, they are often tailored to specific equations. Recent works explore diverse architectures

Read the complete text