Defining Foundation Models for Computational Science: A Call for Clarity and Rigor
Youngsoo Choi, Siu Wun Cheung, Youngkyu Kim, Ping-Hsuan Tsai, Alejandro N. Diaz, Ivan Zanardi, Seung Whan Chung, Dylan Matthew Copeland, Coleman Kendrick, William Anderson, Traian Iliescu, Matthias Heinkenschloss · 2025
Read Defining Foundation Models for Computational Science: A Call for Clarity and Rigor by Youngsoo Choi, Siu Wun Cheung, Youngkyu Kim, Ping-Hsuan Tsai, Alejandro N. Diaz, Ivan Zanardi, Seung Whan Chung, Dylan Matthew Copeland, Coleman Kendrick, William Anderson, Traian Iliescu, Matthias Heinkenschloss (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-22904.txt.
Introduction
Youngsoo Choi and eleven co-authors demand a clear definition of foundation models in computational science. Their 2025 position paper argues the term is drifting. Borrowed from language and vision AI, it now labels many unlike things. Loose use breeds confusion and weak claims.
The authors ground a definition in generality, reusability, and scalability. They list traits such models must show, and traits that are merely nice. Classic finite element methods serve as the yardstick for what foundational means. A new data-driven finite element framework shows one honest path. It weds modular classical structure to learned representations. Rigor, they argue, will keep the label worth having.
Catalog record
| CREATOR | Youngsoo Choi, Siu Wun Cheung, Youngkyu Kim, Ping-Hsuan Tsai, Alejandro N. Diaz, Ivan Zanardi, Seung Whan Chung, Dylan Matthew Copeland, Coleman Kendrick, William Anderson, Traian Iliescu, Matthias Heinkenschloss |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-22904
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Defining Foundation Models for Computational Science: A Call for Clarity and Rigor Youngsoo Choi, Siu Wun Cheung, Youngkyu Kim, Ping-Hsuan Tsai, Alejandro N. Diaz, Ivan Zanardi, Seung Whan Chung, Dylan Matthew Copeland, Coleman Kendrick, William Anderson, Traian Iliescu, Matthias Heinkenschloss Abstract The widespread success of foundation models in natural language processing and computer vision has inspired researchers to extend the concept to scientific machine learning and computational science. However, this position paper argues that as the term “foundation model” is an evolving concept, its application in computational science is increasingly used without a universally accepted definition, potentially creating confusion and diluting its precise scientific meaning. In this paper, we address this gap by proposing a formal definition of foundation models in computational science, grounded in the core values of generality, reusability, and scalability. We articulate a set of essential and desirable characteristics that such models must exhibit, drawing parallels with traditional foundational methods, like the finite element and finite volume methods. Furthermore, we introduce the Data-Driven Finite Element Method (DD-FEM), a framework that fuses the modular structure of classical FEM with the representational power of data-driven learning. We demonstrate how DD-FEM addresses many of the key challenges in realizing foundation models for computational science, including scalability, adaptability, and physics consistency. By bridging traditional numerical methods with modern AI paradigms, this work provides a rigorous foundation for evaluating and developing novel approaches toward future foundation models in computational science. 1 Introduction In recent years, there has been a surge of interest in applying machine learning (ML) and artificial intelligence (AI) to scientific and engineering domains, particularly through the lens of scientific machine learning (SciML). Alongside this growth, an alarming trend has emerged: an increasing number of researchers are labeling their models as foundation models for science and engineering—often without sufficient justification. This trend is understandable. The success of large language models (LLMs), vision transformers, and multi-modal AI systems—commonly referred to as foundation models—has reshaped modern AI bommasani2021opportunities . These models demonstrate broad generalization across tasks and domains, exhibit emergent capabilities, and are pretrained at scale using self-supervised learning. It is natural for researchers to want to associate their models with this success. However, the term ‘foundation model’ carries significant scientific weight and responsibility, especially when used in a research context. Unfortunately, in the computational science and engineering (CSE) community, the use of the term has become ambiguous and inconsistent. Many papers now adopt the label “foundation model” without demonstrating the essential characteristics that would justify such a designation—such as the ability to generalize across diverse physical systems, adapt to new domains or boundary conditions with minimal or no retraining, or extrapolate meaningfully in time and space. This lack of rigor in terminology poses several risks: \bullet It dilutes the meaning of the term, eroding its value for researchers, reviewers, and practitioners. \bullet It introduces confusion in benchmarking and evaluation, especially when models are presented without proper context or generalization metrics. \bullet It undermines scientific rigor, particularly in fields that depend on reproducibility, formal guarantees, and physical validity. In contrast, the traditional computational science community has long used the term foundational methods to describe techniques like the finite element method (FEM) or finite volume method (FVM)—algorithms that are domain-agnostic, broadly applicable, and mathematically grounded. These methods are not data-driven, and thus not considered foundation models in the ML sense. However, they do offer a valuable reference point: a truly foundational tool in science and engineering should work across a wide range of problems with minimal reconfiguration. By analogy, a foundation model in computational science should demonstrate similar breadth—but in a data-driven setting. Therefore, we take the position that the term “foundation model” is being inconsistently and prematurely applied in computational science, and we argue that what is currently missing is a precise, field-appropriate definition of what constitutes a foundation model in computational science. Without it, the research community lacks a shared framework for evaluating claims of generality and adaptability. This paper aims to fill that gap by: \bullet Proposing a rigorous definition of foundation models tailored to computational science; \bullet Outlining a set of core characteristics that foundation models must exhibit; \bullet Drawing connections between traditional foundational methods and modern AI-based foundation models; \bullet Helping authors, reviewers, and practitioners responsibly evaluate and categorize foundation models. Our hope is that this framework will serve as a reference point for future work—encouraging clarity, consistency, and scientific integrity in a rapidly evolving field. We do not claim that the definition proposed in this paper is the only possible one. Rather, we hope it sparks further discussion and inspires alternative, well-reasoned definitions of foundation models tailored to computational science. What matters most is that these definitions emerge with a shared commitment to rigor, transparency, and scientific grounding. Finally, to demonstrate the practical relevance of our proposed definition, we introduce a framework called the data-driven finite element method (DD-FEM), which is inspired by the traditional finite element method—one of the most well-established foundational methods in computational science. DD-FEM is designed to satisfy the key characteristics of a foundation model as outlined in this paper, and serves as a concrete example of how such models may be realized in practice. Alternative Views : Several recent works have referred to their methods, such as neural operators and transformer-based surrogates, as foundation models in computational science. While these approaches demonstrate promising capabilities, such as solver acceleration, domain adaptation, or performance on specific partial differential equation (PDE) families and represent significant advancements in their respective areas, they often fall short of satisfying the broader criteria we articulate in this paper. For example, Deep Operator Network (DeepONet) lu2021learning has been effectively used as a neural preconditioner for Helmholtz equations lee5054726ultra , yet it does not function as a standalone general-purpose solver. Similarly, a pretrained Fourier Neural Operator (FNO) li2020fourier ; pathak2022fourcastnet has been shown to transfer across different PDEs with fine-tuning subramanian2023towards , but the adaptation cost remains high and the generalization narrow. Poseidon, a vision-transformer-based model rabeh2024geometry , and other neural network architectures batatia2024foundation ; mendez2024mole ; marcato2024developing ; jakubik2023foundation show promise in specific geometries or physics regimes, but lack demonstrations of reuse across tasks, boundary conditions, or domain topologies. These contributions are important steps forward, but they highlight the need for a clear, principled definition, one that distinguishes between high performing surrogates and models capable of broad, scalable, and reusable generalization. Our work aims to fill this gap. We elaborate this in more detail in Section 9 . 2 A Definition of Foundation Models in Computational Science Examples of well-known foundation models include language models such as GPT (Generative Pretrained Transformer) vaswani2017attention ; brown2020language ; kalyan2024survey ; bubeck2023sparks , which have revolutionized natural language processing by capturing linguistic patterns and contexts to enable diverse downstream tasks such as text generation, translation, and sentiment analysis. In computer vision, certain models, particularly those based on large-scale pretraining and self-supervised learning, have emerged as foundation models. Notable instances include DINO (Self- di stillation with no labels) caron2021emerging , an example of self-supervised vision transformer, and SAM (Segment Anything Model) kirillov2023segment , both of which demonstrate strong generalization across visual tasks like classification, segmentation, and retrieval with minimal task-specific fine-tuning. Multi-modal models, such as