OmniFlow : A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning
Ruobing Xie, Ruijian Gou, Yuxuan Liang, Xiaomeng Huang, Xian Wu, [0.35em] Tsinghua University Tencent Shenzhen Loop Area Institute, [0.15em] Equal contribution Corresponding author[0.15em] Main Contact: [email protected] · 2026
Read OmniFlow : A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning by Ruobing Xie, Ruijian Gou, Yuxuan Liang, Xiaomeng Huang, Xian Wu, [0.35em] Tsinghua University Tencent Shenzhen Loop Area Institute, [0.15em] Equal contribution Corresponding author[0.15em] Main Contact: [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15797.txt.
Introduction
Wu and co-authors ground image-language models in fluid physics to cut false claims. Language models reason well in words but stumble on PDE-governed flows. Fine-tuning per domain costs much and blocks transfer. Their OMNIFLOW design keeps the base model frozen. A semantic-symbolic step turns flow tensors into topology words. A physics-guided chain injects rules like mass conservation. Reflexive checks then verify each reasoning pass. Tests span micro turbulence, Navier-Stokes, and global weather. Zero-shot and few-shot scores beat deep baselines.
Catalog record
| CREATOR | Ruobing Xie, Ruijian Gou, Yuxuan Liang, Xiaomeng Huang, Xian Wu, [0.35em] Tsinghua University Tencent Shenzhen Loop Area Institute, [0.15em] Equal contribution Corresponding author[0.15em] Main Contact: [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38984 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.15797
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OmniFlow : A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning Ruobing Xie, Ruijian Gou, Yuxuan Liang, Xiaomeng Huang, Xian Wu, [0.35em] Tsinghua University Tencent Shenzhen Loop Area Institute, [0.15em] Equal contribution Corresponding author[0.15em] Main Contact: [email protected] Abstract Large Language Models (LLMs) have demonstrated exceptional logical reasoning capabilities but frequently struggle with the continuous spatiotemporal dynamics governed by Partial Differential Equations (PDEs), often resulting in non-physical hallucinations. Existing approaches typically resort to costly, domain-specific fine-tuning, which severely limits cross-domain generalization and interpretability. To bridge this gap, we propose OmniFlow , a neuro-symbolic architecture designed to ground frozen multimodal LLMs in fundamental physical laws without requiring domain-specific parameter updates. OmniFlow introduces a novel Semantic-Symbolic Alignment mechanism that projects high-dimensional flow tensors into topological linguistic descriptors, enabling the model to perceive physical structures rather than raw pixel values. Furthermore, we construct a Physics-Guided Chain-of-Thought (PG-CoT) workflow that orchestrates reasoning through dynamic constraint injection (e.g., mass conservation) and iterative reflexive verification. We evaluate OmniFlow on a comprehensive benchmark spanning microscopic turbulence, theoretical Navier-Stokes equations, and macroscopic global weather forecasting. Empirical results demonstrate that OmniFlow significantly outperforms traditional deep learning baselines in zero-shot generalization and few-shot adaptation tasks. Crucially, it offers transparent, physically consistent reasoning reports, marking a paradigm shift from black-box fitting to interpretable scientific reasoning. Our code is available at https://github.com/Alexander-wu/OMNIFLOW . 1 Introduction Large Language Models (LLMs) 3 ; 23 ; 46 have demonstrated exceptional symbolic reasoning, code generation, and mathematical problem-solving capabilities. However, when applied to the physical world governed by Partial Differential Equations (PDEs) 4 ; 9 , particularly systems involving continuous spatiotemporal dynamics 44 ; 22 such as turbulence evolution 5 ; 25 or global weather forecasting 29 ; 12 ; 38 ; 10 , LLMs often exhibit significant limitations. Lacking physical grounding, existing multimodal models struggle to comprehend the topological structures within high-dimensional fluid data, frequently resulting in non-physical hallucinations that violate fundamental physical common sense. Figure 1: Comparison of reasoning paradigms. Traditional models (top) are non-interpretable black-boxes. General VLMs (middle) suffer from physical hallucinations. OmniFlow (bottom) integrates a Symbolic Lens and Consistency Loop to deliver grounded forecasts and expert reports, uniting numerical precision with logical reasoning. Historically, two main paradigms have addressed this challenge. ❶. The first involves specialized deep learning models (e.g., FNO 18 , GraphCast 16 ), which serve as surrogates for numerical solvers. While accurate, these models function primarily as data fitters, lacking cross-domain generalization capabilities and interpretability. ❷. The second paradigm involves fine-tuning LLMs with large-scale scientific data. However, fine-tuning incurs high computational costs, often leads to catastrophic forgetting of general knowledge, and still fails to guarantee strict adherence to physical conservation laws, such as mass or momentum conservation 8 . As illustrated in Figure 1 , even advanced Vision-Language Models (VLMs) fall into this second category’s trap: they interpret scientific imagery as semantic patterns rather than discrete solutions to PDEs, leading to visually plausible but physically invalid outputs. We argue that the inability of LLMs to handle physical problems stems not from a lack of internal knowledge, but from a lack of modal alignment and logical constraint. Existing general-purpose VLMs fundamentally lack the numerical precision and physical inductive bias required for spatiotemporal fluid forecasting. Consequently, relying solely on standard VLMs leads to prognostic outputs that strictly violate conservation laws. To address this, we explore a novel paradigm: instead of modifying model parameters, we design a Cognitive Architecture to decouple physical computation from cognitive reasoning, aligning frozen LLMs with rigorous physical rules via neuro-symbolic collaboration. To this end, we propose OmniFlow , a physics-grounded agent framework for generalized fluid dynamics reasoning. As shown in Figure 1 , OmniFlow abandons the traditional black-box prediction mode 27 ; 18 ; 1 ; 39 in favor of a transparent reasoning workflow. First, addressing the heterogeneity of multimodal inputs 6 ; 19 , we design a Visual Symbolic Projector (the Symbolic Lens in Figure 1 ). This module translates raw flow fields (e.g., satellite typhoon imagery) into semantic tokens containing vector field features and topological skeletons, achieving alignment between continuous data and discrete symbols. Second, we introduce a Physics-Guided Chain-of-Thought (PG-CoT) 35 . Within the reasoning engine, the agent operates an In-Context Reflexive Loop (the Consistency Loop in Figure 1 ): it dynamically retrieves external physical knowledge (e.g., Navier-Stokes equations) and executes a Consistency Check during generation 17 . Upon detecting a physical violation (e.g., a trajectory violating inertial constraints), a Critic module forces the model to roll back and self-correct. We ground agentic reasoning in physical laws rather than linguistic plausibility, using symbolic verification to detect and rectify invalid reasoning steps. The main contributions of this paper are as follows: 1. Architectural Innovation : We propose the first VLLM training-free framework for generalized fluid physical reasoning. Through a neuro-symbolic mechanism, OmniFlow successfully activates the reasoning potential of frozen LLMs for complex scientific computing tasks without costly parameter updates. 2. Generalization : We evaluate OmniFlow on three distinct physical benchmarks spanning microscopic turbulence, theoretical Navier-Stokes equations, and macroscopic global weather forecasting. Experiments demonstrate that in a Zero-Shot setting, OmniFlow not only adapts to different governing equations but also achieves prediction accuracy comparable to specialized deep learning models. 3. Interpretability : Unlike the mute numerical outputs of traditional methods, OmniFlow generates structured analysis reports containing physical grounding, risk assessment, and decision logic, providing a new interaction paradigm for scientific discovery and decision support. 2 Related Work Deep Learning for Fluid Dynamics Deep learning has emerged as a powerful paradigm for accelerating fluid dynamics simulations 15 ; 2 , traditionally reliant on computationally expensive numerical solvers. Early data-driven approaches utilized Convolutional Neural Networks (CNNs) to approximate flow fields on fixed grids 31 ; 13 ; 28 . More recently, Physics-Informed Neural Networks (PINNs) 27 have introduced a paradigm shift by embedding Partial Differential Equations (PDEs) directly into the loss function, enabling mesh-free solving. Furthermore, Neural Operators, such as DeepONet 21 and the Fourier Neural Operator (FNO) 18 , have been developed to learn mappings between infinite-dimensional function spaces, achieving resolution-invariant predictions. However, despite their computational efficiency, these surrogate models predominantly operate as black boxes . They excel at numerical regression mapping initial conditions to future states, but lack the capability for explicit symbolic reasoning. Consequently, they cannot articulate the physical mechanisms driving the flow evolution or self-diagnose failures when predictions violate fundamental conservation laws in out-of-distribution scenarios. Multimodal Foundation Models and Scientific Alignment The rapid evolution of Multimodal Large Language Models (MLLMs) 43 has bridged the gap between visual perception and linguistic reasoning. Foundation models like CLIP 26 and LLaVA 19 utilize Vision Transformers (ViT) 7 to align visual features with semantic text embeddings, enabling impressive performance on general visual reasoning tasks. In the scientific domain, specialized architectures such as FourCastNet 14 and Pangu 1 have adapted transformer mechanisms to process atmospheric and fluid data. Nonetheless, directly applying general-purpose vision encoders to fluid dynamics remains challenging. Unlike natural images, fluid imagery (e.g., satellite flow fields) encodes rigorous vector field properties and topological invariants (e.g., vortices and stagnation points) rather than mere texture or object semantics. Existing tokenizers often fail to preserve these continuous physical features during discretization. To address this, our Visual Symbolic Projector is designed to explicitly translate