FEABench: Evaluating Language Models on Multiphysics Reasoning Ability
Nayantara Mudur, Hao Cui, Subhashini Venugopalan, Paul Raccuglia, Michael P. Brenner, Peter Norgaard · 2025
Read FEABench: Evaluating Language Models on Multiphysics Reasoning Ability by Nayantara Mudur, Hao Cui, Subhashini Venugopalan, Paul Raccuglia, Michael P. Brenner, Peter Norgaard (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06260.txt.
Description
Nayantara Mudur and co-authors present a benchmark for language models on physics based engineering tasks in a 2025 paper. Precise simulation is essential in science and engineering, yet models must reason and run solver software end to end. Their benchmark asks models to read problem statements in plain words and operate a commercial finite element program to compute answers.
A full scoring scheme checks each stage of that process. The team also built an agent that calls the software through its interface, inspects outputs, and refines its work over rounds. The best strategy produced runnable calls eighty eight percent of the time. Readers see what it would take for language models to automate real engineering work.
Catalog record
| CREATOR | Nayantara Mudur, Hao Cui, Subhashini Venugopalan, Paul Raccuglia, Michael P. Brenner, Peter Norgaard |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-06260