Residuals Are Not Enough: Limits of Physics-Informed Pre-Training for Scientific Foundation Models
Serge Kotchourko, Amin Totounferoush, Michael W. Mahoney, Steffen Staab · 2025
Read Residuals Are Not Enough: Limits of Physics-Informed Pre-Training for Scientific Foundation Models by Serge Kotchourko, Amin Totounferoush, Michael W. Mahoney, Steffen Staab (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-19081.txt.
Description
Serge Kotchourko, Amin Totounferoush, Michael Mahoney, and Steffen Staab wrote this 2025 study on training large science models with physics rules built in. Such models aim to learn how physical systems behave so they can transfer across tasks. One training trick feeds the model the leftovers of physics equations as a teaching signal, and the team asked whether that trick helps a broad model, not just a narrow one.
Tests ran across many equation types, from simple setups to fluid flow and odd shapes. The physics-based training clearly helped in neat, well-matched settings, where it cut the need for data. But the gains faded as tasks grew harder or drifted from the training setup. With new boundary rules or new equation types, the physics-trained models did no better than plain ones, and at times worse.
Catalog record
| CREATOR | Serge Kotchourko, Amin Totounferoush, Michael W. Mahoney, Steffen Staab |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-19081