A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptions

Elisa Negrini, Yuxuan Liu, Liu Yang, Stanley Osher, Hayden Schaeffer · 2025

Read A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptions by Elisa Negrini, Yuxuan Liu, Liu Yang, Stanley Osher, Hayden Schaeffer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-06026.txt.

Introduction

Elisa Negrini and co-authors build a foundation model that solves differential equations and describes them in words. Such equations drive surrogate models, live forecasts, and control tasks. Past foundation solvers handled numbers or symbols, never plain text as well. Their transformer takes equation settings, start values, and prose sketches of the physics together.

The model outputs both the numeric solution and a written account of its behavior. In-family tests erred under 3.3 percent on average. Fresh families still stayed under 7.8 percent error. Every generated description in the tests came out correct. Time extrapolation beyond the training span also worked in select trials.

Catalog record

CREATORElisa Negrini, Yuxuan Liu, Liu Yang, Stanley Osher, Hayden Schaeffer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-06026

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptions Elisa Negrini, Yuxuan Liu, Liu Yang, Stanley Osher, Hayden Schaeffer Abstract Neural networks are one tool for approximating non-linear differential equations used in scientific computing tasks such as surrogate modeling, real-time predictions, and optimal control. PDE foundation models utilize neural networks to train approximations to multiple differential equations simultaneously and are thus a general purpose solver that can be adapted to downstream tasks. Current PDE foundation models focus on either learning general solution operators and/or the governing system of equations, and thus only handle numerical or symbolic modalities. However, real-world applications may require more flexible data modalities, e.g. text analysis or descriptive outputs. To address this gap, we propose a novel multimodal deep learning approach that leverages a transformer-based architecture to approximate solution operators for a wide variety of ODEs and PDEs. Our method integrates numerical inputs, such as equation parameters and initial conditions, with text descriptions of physical processes or system dynamics. This enables our model to handle settings where symbolic representations may be incomplete or unavailable. In addition to providing accurate numerical predictions, our approach generates interpretable scientific text descriptions, offering deeper insights into the underlying dynamics and solution properties. The numerical experiments show that our model provides accurate solutions for in-distribution data (with average relative error less than 3.3%) and out-of-distribution data (average relative error less than 7.8%) together with precise text descriptions (with correct descriptions generated 100% of times). In certain tests, the model is also shown to be capable of extrapolating solutions in time. 1 Introduction † † footnotetext: Code for this work is available at https://github.com/enegrini/MOL-LLM.git . Neural networks have become increasingly important for solving non-linear differential equations, with applications in climate modeling, financial forecasting, biological systems analysis, and structural optimization (see for instance [ 32 , 36 , 8 ] ). Their ability to model complex, non-linear relationships allows for efficient and accurate predictions for various scientific computing tasks such as surrogate modeling, real-time predictions, and optimal control. Previous work in deep learning for partial differential equations (PDE) have focused on learning either the solution operator, which maps input functions to their solutions, or the governing system of equations, which describes the constitutive model based on observations of state variables [ 6 , 22 , 21 , 29 , 51 , 35 ] . These approaches, however, tackle one task at a time and are limited to the use of numerical data. Building on the observation that families of differential equations frequently share fundamental characteristics, recent work has introduced transformer-based architectures to enable simultaneous encoding of various parametric differential equations [ 46 , 47 , 48 , 5 , 25 , 39 , 24 , 17 ] . Although effective, these methods require structured input and output data, with vanilla ICON [ 46 ] focusing on numerical data and PROSE utilizing numerical and symbolic data [ 25 , 39 ] . In the work, we consider additional modalities as both inputs and outputs to the model. Often, one has access or would like to produce heuristic descriptions of the observed dynamics that are neither in symbolic nor numerical form, but instead come as textual descriptions. For example, consider modeling the dynamics of a complex ecological system: the numerical inputs can include measured population levels, while textual inputs could describe key processes such as predator-prey interactions or migration patterns. Similarly, in material science, numerical data could represent experimental results, while textual inputs provide the governing equation or describe the experimental setup. By combining these modalities, the model may better capture the underlying rules and provide more accurate and contextually informed predictions. The utilize of mathematical formulae, text descriptions, and numerical values can provide a more robust development toward a PDE foundation model. Note that in ICON-LM [ 47 ] both textual and numerical prompts were provided as inputs; however, the model does not generate text descriptions since the outputs of ICON-LM are the numerical predictions. Consider the following parametrized differential equation: \displaystyle\begin{cases}\mathcal{F}(u(x,t;c))=0\quad(x,t)\in\Omega\times[0,T]\\ \mathcal{B}(u(x,t;c))=0\quad(x,t)\in\partial\Omega\times[0,T]\\ u(x,0;c)=\mathcal{G}(x;c),\quad x\in\Omega,;c\sim\mathcal{D}\end{cases} (1) where \mathcal{F} denotes the governing equation, \mathcal{B} denotes boundary conditions. The initial condition \mathcal{G} is a generating function, and c denotes the parameters that determine the initial conditions from distribution \mathcal{D} . The objective is to develop a single neural network model capable of approximating numerically and describing in text the solution operators for a range of governing equations \mathcal{F} in ( 1 ), which can include ODEs and PDEs. We will assume periodic boundary conditions for uniformity in experiments. The text input may contain either the equation to solve or a textual description of the system. This is important in applications where one may only have a partial model or a description of the underlying process. The numerical data includes the equation’s parameters and initial conditions. For instance, suppose we want to solve the heat equation with parameter c=0.003 and initial condition u(x,0)=u_{0}(x) . The input to our model could be: • A formal description: “The given equation is u_{t}=cu_{xx} where c=0.003 and u(x,0)=u_{0}(x) .” • A more general description: “Solve the heat equation with parameter 0.003 and initial condition u_{0}(x) .” In our experiments, we primarily used the first type of input, where the equation is explicitly provided as a formula. However, given sufficient training examples, we expect the model to also perform well when only a descriptive name or general description of the equation is provided. With the input “The given equation is u_{t}=cu_{xx} where c=0.003 and u(x,0)=u_{0}(x) ,” our model outputs: 1. Numerical Predictions : The solution to the PDE at user-defined query locations. 2. Scientific Text Descriptions : A sentence or more that describes the scientific properties of the equation or the solution. For instance: • Physical process modeled by the equation: “The heat equation is a parabolic PDE that models the spread of heat in a material over time.” • Alternative numerical methods to approximate the solution: “The heat equation can be numerically solved using a combination of forward-time and central-space finite difference methods.” The generality of the generated text descriptions depends on the training set. For instance, descriptions of other properties of interest may be included in the training set, such as the long-term behavior of solutions, the presence of shocks or rarefactions, etc. We refer to Section 4.4 , Table 5 for various examples of generated text. The text is collaboratively trained in the neural network with the goal of generalizing text descriptions to new systems. As a representative example of our model’s capabilities, in Section 4 we show that our model can determine whether the solution to a given conservation law from the conservation laws dataset will exhibit no shocks, shocks, or rarefactions. Specifically, on this dataset the numerical output achieves a test error of 1.41% (see Table 2 ). In Figure 2 , the last three rows illustrate that, when present, the locations and intensities of rarefactions and shocks are correctly identified. Finally, Table 5 shows that the text descriptions accurately capture shocks and rarefactions when present, with an F1 score greater than 0.94. 1.1 Main Contributions. We proposes a novel multimodal framework that integrates numerical and textual input and output data for PDE foundation modeling. Our

Read the complete text