LLMs with in-context learning for Algorithmic Theoretical Physics

2026

Read LLMs with in-context learning for Algorithmic Theoretical Physics (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08212.txt.

Introduction

Anamaria Hell and Leander Thiele test whether a large language model, wired to a computer algebra system, can do routine algorithmic work in theoretical physics. Such computations are conceptually simple but slow and full of traps, so reliable automation would save researchers real time.

The authors connect Claude to the Maple algebra system and aim the pair at cosmological perturbations in modified gravity theories. Given worked examples as context, the frontier model solved most of the test problems. The study catalogs the typical failure modes that remain and shows which ones extra context can fix, giving a grounded picture of where language models help symbolic physics today.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08212

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LLMs with in-context learning for Algorithmic Theoretical Physics Abstract There is an increasing number of algorithmic computations in theoretical physics. These, while conceptually simple, can nevertheless be time-consuming and contain subtleties that should not be overlooked. Given the recent improvement of Large Language Models (LLM), it is natural to investigate whether LLMs equipped with a computer algebra system (CAS) runtime and sufficiently informative context can reliably carry out these algorithmic tasks. In this work, we interface Claude with Maple, and apply this framework to cosmological perturbations in modified theories of gravity. We demonstrate the current capabilities of this approach, the typical failures, and how the same can be improved. We find that a frontier LLM supplied with worked examples is able to solve most test problems. † † affiliationnotice: IPMU26-0020 Anamaria Hell and Leander Thiele Kavli IPMU (WPI), UTIAS, The University of Tokyo, and Center for Data-Driven Discovery, 5-1-5 Kashiwanoha, Kashiwa, Chiba 277-8583, Japan 1 Introduction Theoretical physicists deal with a range of tasks in their research. Between the most mechanistic calculations which have been relegated to numerical and symbolic computer programs, and the art of constructing new theories born out of genuine creativity, lies a large field of “algorithmic computations”. By this we mean tasks that are sufficiently complex that a general solution in terms of a deterministic computer program is unavailable, while at the same time not being of such difficulty that entirely new methods and ways of thinking are required for their solution. To bracket algorithmic computations by examples, we mean tasks more complex than individual commands in a computer algebra system, but less complex than the discovery of AdS-CFT correspondence or certain proofs in Einstein gravity. This leaves a large terrain of computations for which a generic recipe exists while the details vary from problem to problem and certain subtleties may require special care. Perturbative calculations (like scattering matrices) in renormalizable QFT, string theory, or EFTs are typical examples. More familiar to general physicists would be typical problems in classical electrodynamics or perturbation theory in quantum mechanics. Such algorithmic computations currently take up much time for many theoretical physicists, even with the help of computer algebra systems (CAS). At the same time, their relative simplicity implies that their solution using agentic AI might be within reach. The algorithmic nature and frequency of these problems implies that a large set of worked examples is available and an abstract description of the algorithm can be written down. In this work, we use both worked examples and the algorithm’s description for in-context learning and test which contents in the context optimize success rate. We thus test in this work the ability of a frontier large language model, interfaced with a CAS session and equipped with an informative context, to solve problems from a specific class of algorithmic computations: identifying the degrees of freedom for cosmological backgrounds in various theories of gravity . Cosmological perturbations, small ripples in spacetime and matter, give rise to the structure of the Universe that we observe today ( 37 ; 32 ; 7 ; 25 ; 36 ; 33 ) . They represent the degrees of freedom, the building blocks of theories, which determine their stability and observational consequences. Identifying them is thus essential, and can be non-trivial when departing from Einstein’s General Relativity. Several approaches have been developed with this purpose, including the Hamiltonian analysis with a Dirac-Bergmann algorithm ( 4 ; 10 ; 3 ) , and a Faddeev-Jackiw approach ( 13 ) , as well as Lagrangian formalism (see 20 for a general prescription, and references therein), and a more rigorously mathematical approach founded directly in the equations of motion (see 18 and references therein). In this work, we follow the recipe given in 20 , which allows for the easiest and most straight-forward implementation in the context of cosmological perturbations. For the purposes of a rigorous study, algorithmic computations of cosmological perturbations have some distinct advantages. It is easy to construct test problems with research-grade complexity and with such obscure features that they are impossible to have been encountered in the LLMs training set. In fact, in our evaluation we sometimes observe the LLM attempting to relate the task to some better-known problem from the literature. At the same time, the type of problems have unique and easily checked solutions. Our work aims to establish how much theoretical physicists can benefit from automated computation specifically in algorithmic problems. To this end, we answer the following questions: 1. What are current capabilities of frontier LLMs equipped with a CAS runtime; 2. How should the LLM’s context be used; 3. What are typical failure modes and how can they be mitigated. Thanks to the restricted set of problems and our available expert knowledge, we can understand the LLM’s solution strategies beyond simply checking the final answer, thus enabling a complementary qualitative evaluation. A central aim of this work is to develop practical guidelines for how practitioners can employ their existing worked examples as useful context for LLMs to autonomously tackle similar problems. As we demonstrate in Sec. 2 , most work on LLMs equipped with a symbolic computation tool has concentrated on mathematics. Compared to these works, we focus less on the model’s creativity in finding proof strategies and more on its ability to perform relatively routine computations that are simply burdensome for the human physicist. Note, however, that these computations are not necessarily easy, and the use of a CAS does not guarantee correctness. Another unusual aspect of this work is the difficulty in defining a locally relevant context: some manipulations need to be carried out with later steps in mind (for example, solution of background equations may produce roots if done naively, which would lead to problems in subsequent steps). Thus, algorithmic theoretical physics is an interesting testing ground for long contexts. The remainder of this paper is structured as follows. In Sec. 2 , we discuss previous works applying LLMs equipped with CAS to mathematics and physics problems. In Sec. 3 , we describe our setup, experiments, and evaluation strategy. In Sec. 4 , we summarize our results both qualitatively and quantitatively. We conclude in Sec. 5 . 2 Related Work Using structure in an LLM’s response goes back to chain-of-thought ( 39 ) . A natural extension of this idea is to interface the model with tools ( 15 ; 6 , e.g.,) . Such tool-aided reasoning has found widespread application in mathematics, thanks to the strict verifiability afforded by programs such as Lean ( 17 ; 24 ; 5 ; 34 ; 30 , e.g.,) . Not surprisingly, tool use is found to be especially helpful for more difficult problmes ( 9 ) . In a similar fashion to this work, in-context learning ( 11 , e.g.,) has been studied as a way to improve the LLM’s performance in math problems ( 27 ; 38 ; 14 , e.g.,) . In contrast to explicitly fine-tuned models, general purpose models with useful context are easier to use in a human-AI collaboration and can be more easily adapted if features in the tools (e.g., Lean strategies) change ( 5 ) . In the context of theoretical physics, using LLMs

Read the complete text