PhyDrawGen: Physically Grounded Diagram Generation from Natural Language

Nafiul Haque, Syed Nazmus Sakib, Shifat E Arman · 2026

Read PhyDrawGen: Physically Grounded Diagram Generation from Natural Language by Nafiul Haque, Syed Nazmus Sakib, Shifat E Arman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30512.txt.

Introduction

Nafiul Haque, Syed Nazmus Sakib and Shifat E Arman present PhyDrawGen, a hybrid flow splitting scene meaning from rule checks, starting from a typed scene graph. A fixed solver then turns this graph into a Planar Straight-Line Graph (PSLG), holding force balance, optical paths, and field shapes. Finally, a fine-tuned Qwen-VL model runs a propose-verify loop to correct rule breaks. Tested on 1,449 problems spanning motion, optics, and EM, PhyDrawGen clearly beats GPT-5-image, Gemini 2.5 Flash, and Gemini 3 Pro.

Catalog record

CREATORNafiul Haque, Syed Nazmus Sakib, Shifat E Arman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66162
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30512

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PhyDrawGen: Physically Grounded Diagram Generation from Natural Language Nafiul Haque, Syed Nazmus Sakib, Shifat E Arman Abstract Generating physics diagrams from text requires strict adherence to physical laws. While current generative models produce visually plausible outputs, they systematically hallucinate force vectors, ignore conservation laws, and violate geometric constraints. We present PhyDrawGen , a neuro-symbolic pipeline that decouples semantic scene understanding from physical constraint satisfaction. First, a large language model extracts a typed scene graph from the problem text. A deterministic solver then converts this graph into a Planar Straight-Line Graph (PSLG), encoding force balance, optical paths, and field topologies as exact geometric primitives. Finally, a fine-tuned Qwen-VL model implements a visually grounded propose-verify loop to iteratively correct any constraint violations. Evaluated on a benchmark of 1,449 problems spanning mechanics, optics, and electromagnetism, PhyDrawGen significantly outperforms GPT-5-image, Gemini 2.5 Flash, and Gemini 3 Pro, demonstrating robust physical accuracy even on unusual-object problems. 1 Introduction Physics diagrams, free-body diagrams, ray-optics constructions, and electromagnetic field maps are formal visual arguments in which every arrow encodes a physical law, every angle encodes a geometric constraint, and every spatial relationship encodes an interaction governed by classical physics. A force arrow pointing in the wrong direction is not merely an aesthetic defect; it is a false physical claim. As such the generation of physically grounded scientific diagrams represents a frontier at the intersection of natural language understanding, structured reasoning, and constrained visual synthesis. Though diffusion models are capable of producing photorealistic images ( Rombach et al. 2022 ; Ramesh et al. 2022 ; Saharia et al. 2022 ; Ho et al. 2020 ) and spatially conditioned generation through adapters ( Zhang et al. 2023 ; Mou et al. 2024 ; Ye et al. 2023 ) and grounding mechanisms ( Li et al. 2023 ; Bar-Tal et al. 2023 ; Johnson et al. 2018 ) , the noise-addition and denoising architecture of diffusion models is fundamentally ill-suited to tasks that require hard constraint satisfaction. The denoising process optimizes for perceptual plausibility under a learned prior, not for algebraic correctness under physical law. When applied to physics diagram generation, this manifests as systematic failure: diffusion models hallucinate force directions, place arrows at geometrically inconsistent angles, omit forces that conservation laws require, and conflate visually similar but physically distinct configurations such as static friction opposing motion versus kinetic friction during sliding. Recent benchmarks have documented the remarkable ability of large language models (LLMs) and vision-language models (VLMs) to solve physics problems from diagrams ( He et al. 2024 ; Xiang et al. 2025 ; Lu et al. 2022 ; Yue et al. 2024 ; Lu et al. 2024 ) , demonstrating strong chain-of-thought reasoning over structured visual inputs. Furthermore, LLMs exhibit strong capabilities in structured extraction from natural language, as demonstrated by scene graph generation pipelines ( Gao et al. 2024 ) and spatial reasoning systems ( Li et al. 2023 ) . Crucially, chain-of-thought prompting ( Wei et al. 2022 ; Kojima et al. 2022 ) enables LLMs to decompose physics problems into typed entities and relationships — objects, surfaces, physical actions, forces, and geometric constraints — in a form that is amenable to downstream constraint solving. We leverage this capacity in the opposite direction: rather than parsing a diagram to solve a problem, we parse a problem to construct a diagram. We propose PhyDrawGen , a pipeline that separates the semantic task of understanding a physics problem from the symbolic task of satisfying its physical constraints by employing an LLM as a structured scene graph extractor and a deterministic constraint solver as an exact physical verifier. The constraint solver produces a Planar Straight-Line Graph (PSLG) in which physical laws are encoded as typed geometric primitives that covers mechanics, optics, and electromagnetism under a single representational framework. The complete output is a standard physics diagram comprising a full scene with labeled force arrows and per-object free-body diagrams. This approach addresses the discrepancy between probabilistic visual representations and structural physical laws. By transforming text into symbolic blueprints prior to rendering, we ensure semantic flexibility does not compromise geometric or physical truth. We make three contributions: 1. The PhyDrawGen scene graph schema. A typed heterogeneous graph whose Constraint node vocabulary maps directly to algebraic physical conditions. The schema covers mechanics, optics, and electromagnetism under a unified typed vocabulary of six node classes and six edge relations. 2. The PSLG Constraint Solver. A deterministic analytical solver that converts the scene graph into a Planar Straight-Line Graph encoding force balance, optical ray consistency, and field line topology as typed geometric constraint primitives. 3. A Learned Constraint Correction Loop. A vision-language model fine-tuned via supervised learning on automatically generated constraint-violation examples, implementing a propose-verify correction loop that combines exact symbolic constraint checking with visually-grounded language model correction. 2 Related Work Controlled synthesis and structured generation. While diffusion models and spatial adapters have revolutionized text-to-image synthesis ( Rombach et al. 2022 ; Ramesh et al. 2022 ; Saharia et al. 2022 ; Ho et al. 2020 ; Song et al. 2021 ; Zhang et al. 2023 ; Mou et al. 2024 ; Ye et al. 2023 ) , they inherently optimize for visual plausibility under a learned prior rather than hard symbolic constraints. Consequently, even with advanced layout grounding ( Li et al. 2023 ; Bar-Tal et al. 2023 ) or sketch guidance ( Xing et al. 2026 ; Vinker et al. 2022 ) , these architectures cannot guarantee that generated force arrows or optical rays adhere to Newton’s or Snell’s laws. To enforce physical coherence, recent work has increasingly turned to structured intermediate representations. By conditioning synthesis on typed relational graphs ( Johnson et al. 2018 ; Xu et al. 2017 ; Gao et al. 2024 ) and utilizing neuro-symbolic frameworks that decouple semantic reasoning from deterministic verification ( Huang et al. 2026 ) , systems can achieve robust constraint satisfaction. PhyDrawGen extends this paradigm; we leverage LLM chain-of-thought spatial reasoning ( Li et al. 2023 ; OpenAI 2023 ; Liu et al. 2023 ; Bai et al. 2023 ; Wei et al. 2022 ; Kojima et al. 2022 ) but replace general spatial semantics with a domain-specific vocabulary grounded strictly in classical physics. Figure 1: The overall pipeline of PhyDrawGen. To bridge semantic understanding and algebraic exactness, a language model (GPT-4o) extracts a typed scene graph from the text (left box). A solver converts it into a Planar Straight-Line Graph enforcing physical primitives (middle box), while a Qwen-VL loop iteratively corrects violations before rendering. Physics reasoning and diagram understanding. The ability of vision-language models to reason over physics diagrams has been extensively benchmarked ( He et al. 2024 ; Xiang et al. 2025 ; Lu et al. 2022 ; Yue et al. 2024 ; Lu et al. 2024 ) , revealing strong chain-of-thought performance on diagram-based problem solving across mechanics, optics, and electromagnetism. These benchmarks establish that current models can interpret physics diagrams with high accuracy, yet the inverse capability which is generating physically correct diagrams from natural language has received minor attention. Physics education research has long documented that even advanced students who conceptually understand force and motion frequently fail to construct geometrically correct diagrams ( Vignal and Wilcox 2022 ;

Read the complete text