Physics-Guided Foundation Model for Scientific Discovery: An Application to Aquatic Science

Runlong Yu, Chonghao Qiu, Robert Ladwig, Paul Hanson, Yiqun Xie, Xiaowei Jia · 2025

Read Physics-Guided Foundation Model for Scientific Discovery: An Application to Aquatic Science by Runlong Yu, Chonghao Qiu, Robert Ladwig, Paul Hanson, Yiqun Xie, Xiaowei Jia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-06084.txt.

Introduction

Runlong Yu and co-authors propose a physics-guided foundation model for science, tested on lake water temperature and dissolved oxygen. Machine learning models that respect physical laws have mostly been built for single, simple tasks. Real systems couple many processes and features at once, which limits those designs.

Their model blends pre-trained learning models with physics-based simulators. It is first trained inside a simulated environment that spans many features and variables, learning which interactions matter under several goals at once. Then it is fine-tuned on real observations while keeping laws like conservation of mass and energy. Tests on real lakes show the method works, and the authors claim wide use across science.

Catalog record

CREATORRunlong Yu, Chonghao Qiu, Robert Ladwig, Paul Hanson, Yiqun Xie, Xiaowei Jia
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-06084

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Physics-Guided Foundation Model for Scientific Discovery: An Application to Aquatic Science Runlong Yu, Chonghao Qiu, Robert Ladwig, Paul Hanson, Yiqun Xie, Xiaowei Jia Abstract Physics-guided machine learning (PGML) has become a prevalent approach in studying scientific systems due to its ability to integrate scientific theories for enhancing machine learning (ML) models. However, most PGML approaches are tailored to isolated and relatively simple tasks, which limits their applicability to complex systems involving multiple interacting processes and numerous influencing features. In this paper, we propose a P hysics- G uided F oundation M odel ( PGFM ) that combines pre-trained ML models and physics-based models and leverages their complementary strengths to improve the modeling of multiple coupled processes. To effectively conduct pre-training, we construct a simulated environmental system that encompasses a wide range of influencing features and various simulated variables generated by physics-based models. The model is pre-trained in this system to adaptively select important feature interactions guided by multi-task objectives. We then fine-tune the model for each specific task using true observations, while maintaining consistency with established physical theories, such as the principles of mass and energy conservation. We demonstrate the effectiveness of this methodology in modeling water temperature and dissolved oxygen dynamics in real-world lakes. The proposed PGFM is also broadly applicable to a range of scientific fields where physics-based models are being used. 1 Department of Computer Science, University of Pittsburgh 2 Department of Ecoscience, Aarhus University 3 Center for Limnology, University of Wisconsin-Madison 4 Department of Geographical Sciences, University of Maryland {ruy59,chq29,xiaowei}@pitt.edu, [email protected], [email protected], [email protected] Introduction Physics-based models of dynamical systems are often used to study scientific systems. For instance, scientists in aquatic science build physics-based models to simulate different water quality variables such as water temperature and dissolved oxygen (DO) concentrations, which are vital for assessing ecosystem health and water security. Similar models are also applied in agriculture ( Jia et al. 2019a ) , geology ( Reichstein et al. 2019 ) , climate science ( Faghmous and Kumar 2014 ) , and bio-medicine ( Yazdani et al. 2020 ) . Despite their widespread use, physics-based models face limitations due to the simplified representations of complex physical processes and the challenges of selecting appropriate parameters ( Jia et al. 2019b ) . Additional complexity arises when coupling these models to perform multiple tasks due to the dependencies amongst physical processes, e.g., DO concentrations are highly dependent on water temperature profiles. With advances in data collection driven by improved sensor technologies, there is a growing interest in using machine learning (ML) to extract complex data patterns for scientific problems ( Willard et al. 2022 ; He et al. 2023 ; Wang et al. 2024 ; Xu et al. 2024 ) . However, most ML approaches are only tested for isolated and simple tasks while still being limited in applicability to complex real-world systems with interacting and non-stationary processes. Recently, foundation models have shown great promise in tasks like vision and natural language processing by pre-training over large datasets ( Bommasani et al. 2021 ; Zhou et al. 2023 ; Ye et al. 2024 ) . These models also offer tremendous opportunities for scientific modeling due to their ability to harness large and heterogeneous data and adapt to diverse downstream tasks. Figure 1: Heat energy and DO mass fluxes in the lake. However, direct application of existing foundation models to scientific problems often leads to serious false discoveries due to several major challenges: 1. Data requirements : Advanced ML models can often outperform traditional empirical models (e.g., regression), but these models require extensive training data, which is often scarce in real scientific applications. 2. Effectiveness of pre-training : Unsupervised pre-training has been shown to significantly boost the performance of many existing foundation models and mitigate the need for large data in downstream tasks. However, traditional pre-training tasks are not well aligned with scientific modeling tasks and thus can be less effective. The datasets used to pre-train existing models also have little overlap with target scientific data. 3. Physical consistency and generalizability : With the absence of physical knowledge, existing foundation models can only learn statistical patterns from available data. Although the model could perform well in similar training data distribution, the patterns extracted by ML models may significantly violate some established physical relationships (e.g., mass and energy conservation). Consequently, these models can struggle to generalize to unseen scenarios. 4. Learning multiple tasks : Most existing foundation models for scientific problems still focus on single prediction tasks ( Li et al. 2024 ; Xie et al. 2024 ) while largely ignoring the dependencies among multiple physical variables. This restricts their utility in complex systems characterized by multiple tasks and numerous influencing features. To address these challenges, in this paper, we propose a P hysics- G uided F oundation M odel ( PGFM ) framework. Instead of relying on complex model architecture, PGFM integrates scientific knowledge from existing physics-based models to guide model pre-training and adaptation. Our proposed PGFM is implemented and evaluated in the context of predicting water temperature and DO concentrations in lake systems. Both variables are key indicators of water quality and are intertwined with ecosystem phenology, influenced by features such as morphometric characteristics of lakes, weather conditions, trophic states, and watershed land use ( Read et al. 2019 ; Ladwig et al. 2022 ; Yu et al. 2024a ) . Also, water temperature and DO concentrations are highly interdependent, with temperature shifts affecting oxygen solubility and biochemical reactions. Modeling the dynamics of these water quality variables can provide important insights for resource managers to make informed decisions to ensure safe drinking water, preserve aquatic habitat, and support sustainable water resource management. Ideally, the foundation model should preserve three key properties. First, the model should be generalizable to large and diverse regions, e.g., predicting water quality in different lakes, even when they are sparsely observed. Second, the model should be able to perform multiple prediction tasks related to the target system. Third, the model needs to capture the physical dependencies between different tasks. To achieve these goals, the PGFM framework includes meticulously designed pre-training and fine-tuning stages. In particular, the pre-training is conducted in a simulated environmental system that encompasses a wide range of data features and various simulated variables generated by physics-based models. By using an evolution-based algorithm, the foundation model progressively evolves to select features and their interactions that best reflect the dynamics of multiple simulated variables in the system. When fine-tuning the foundation model to each specific object (i.e., lake) and task (e.g., water temperature or DO concentration for a particular depth layer), we reuse the extracted features obtained from pre-training and also enhance the training objective with physical relationships (e.g., mass and energy conservation) specific to the task. To explicitly capture the interdependence between water temperature and DO concentration in this work, we augment the input for DO modeling with predicted lake temperatures, thereby enhancing the model’s robustness by providing physically relevant information. Our evaluations on a wide range of lakes in the Midwestern USA demonstrate the capability of PGFM to effectively predict water temperature and DO concentration even with limited observed data. Our code is available at https://github.com/RunlongYu/PGFM. Problem Formulation The objective of

Read the complete text