Rethinking Scientific Modeling: Toward Physically Consistent and Simulation-Executable Programmatic Generation *** Conference: The 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2026; Jeju, South Korea CCS: Computing methodologies Artificial intelligence Yongqing Jiang, Jianze Wang, Zhiqi Shen, Zhenghong Lin, Jiayuan Wang, Yijian Yang, Kaoshan Dai *, Haoran Luo * Abstract . Structural modeling is a fundamental component of computational engineering science, in which even minor physical inconsistencies or specification violations may invalidate downstream simulations. The potential of large language models (LLMs) for automatic generation of modeling code has been demonstrated. However, non-executable or physically inconsistent outputs remain prevalent under stringent engineering constraints. A framework for physics-consistent automatic building modeling is therefore proposed, integrating domain knowledge construction, constraint-oriented model alignment, and verification-driven evaluation. CivilInstruct is introduced as a domain-specific dataset that formalizes structural engineering knowledge and constraint reasoning to enable simulation-ready model generation. A two-stage fine-tuning strategy is further employed to enforce constraint satisfaction and application programming interface compliance, substantially reducing hallucinated and non-conforming outputs. MBEval is presented as a verification-driven benchmark that evaluates executability and structural dynamics consistency through closed-loop validation. Experimental results show consistent improvements over baselines across rigorous verification metrics. Our code is available at https://github.com/Jovanqing/AutoBM . 1 1 footnotetext: Corresponding authors 1. Introduction Figure 1. Task formulation of llm-driven automatic building modeling from natural language descriptions Figure 2. Overview of this study, illustrating the integration of large language models with structural engineering knowledge for Automatic Building Modeling (AutoBM), including task motivation, data foundations, and the framework Accurate numerical modeling is a cornerstone of computational structural science. Recent advances in large language models (LLMs) ( 38 ; 40 ; 29 ; 2 ) introduce a new paradigm for scientific modeling. As shown in Figure 1 , LLMs can interpret natural-language specifications and generate corresponding structural modeling code ( 36 ; 15 ; 18 ) , enabling automated construction of simulation-ready models. Within this emerging paradigm, structural modeling stands out as a compelling domain due to its complexity, rule constraints, and reliance on expert knowledge. OpenSeesPy is a cornerstone platform for seismic and structural analysis ( 44 ; 4 ; 35 ; 53 ) due to its strong capability in simulating complex structural systems (Figure 2 (a)). When integrated with OpenSeesPy and multi-source data (Fig. 2 (b)), LLMs enable automatic building modeling (AutoBM) across multiple building types by directly generating modeling code. Relative to rule-based template methods (Fig. 2 (c)), LLM-driven modeling offers superior scalability, positioning it as a promising computational interface for engineering simulation. Despite this promise, existing LLM-based approaches to structural modeling remain fundamentally limited. Automated approaches based on LLMs are increasingly applied to modeling research (e.g., structural verification ( 27 ) , 2D frame modeling ( 12 ) , integration with the computational engine ( 26 ) of OpenSeesPy). These LLM-driven modeling approaches ( 5 ; 8 ; 6 ) have been shown to improve the efficiency of building modeling. However, prior work largely overlooks task-specific fine-tuning for structural engineering, despite the domain’s strict requirements on engineering compliance beyond syntactic correctness. As shown in Figure 2 (d), without domain-specific knowledge integration, ensuring the model’s physical correctness is difficult for general-purpose LLMs. This raises a key question: "How can LLMs be systematically constrained to enable structurally and physically valid programmatic generation for building modeling?" To address this question, we propose a physics-aware LLM framework for AutoBM, as shown in Figure 2 (e). The main contributions of this work can be summarized as follows: • Novel AutoBM Task: We formalize Automatic Building Modeling (AutoBM) as a research task with clearly defined problem boundaries. The task specifies user-driven modeling requirements by defining task inputs (building function, plan dimensions) and task outputs (engineering-compliant, high-precision structural modeling code). • BMBench Dataset: We propose BMBench, a comprehensive AutoBM benchmark integrating large-scale instruction data and multidimensional evaluation metrics across 10,912 samples and 128 cases. • RLA-SPC: We propose a reinforcement learning alignment strategy under structural physical constraints, and experimental results show that RLA-SPC improves LLM compliance with engineering physical constraints. 2. Related Work Large Language Models. General-purpose LLMs ( 1 ; 10 ; 45 ) exhibit limited generalization in specialized domains, prompting domain adaptation efforts and the development of domain-specific benchmarks across medicine ( 37 ) , finance ( 41 ) , and biology ( 30 ) . Within the civil engineering domain, applied research on LLMs exhibits a trend toward multifaceted breakthroughs. For example, Kim et al. ( 22 ) and Pandey et al. ( 32 ) demonstrated the feasibility of using LLMs to generate high-fidelity simulation codes for fluid-structure interaction and computational fluid dynamics (CFD), respectively. Qin et al. ( 33 ) developed an intelligent shear wall design framework that achieved substantial gains in design efficiency while satisfying mechanical constraints. Jiang et al. ( 21 ; 20 ) and Jeoung et al. ( 19 ) applied visual-language modeling techniques to automate visual analytics tasks, including post-earthquake damage assessment and construction site monitoring. Figure 3. The definition and overview of the AutoBM task. Automatic Structural Modeling and Reasoning. Early computational approaches for design–analysis mapping mainly relied on rigid scripting interfaces ( 49 ; 17 ; 28 ) or parametric template-based generation methods ( 14 ; 31 ; 52 ; 7 ) . With the emergence of LLMs, existing studies have explored domain adaptation through model fine-tuning ( 43 ) and the integration of structured domain knowledge (e.g., knowledge graphs ( 46 ; 24 ) , material mechanism discovery ( 16 ) , energy system optimization ( 13 ) ). More recent research has further advanced toward system-level reasoning by adopting multi-agent collaboration mechanisms ( 47 ; 9 ; 23 ) , while the incorporation of chain-of-thought reasoning and literate programming paradigms has enabled LLMs to orchestrate end-to-end simulation pipelines ( 50 ) , motivating the development of domain-specific benchmarks for systematic evaluation of LLM-driven engineering automation ( 3 ; 25 ; 39 ; 11 ) . 3. Task: AutoBM The AutoBM task is formally defined as the process of generating executable, physically consistent structural modeling code from structured engineering specifications using LLMs. As shown in Figure 3 , the fundamental objective of AutoBM is to bridge the gap between natural language intent and rigorous structural engineering requirements while strictly adhering to physical constraints. Formally, a building modeling specification is defined as \mathcal{S}=(D_{T},D_{P},D_{A}) . In this formulation, the textual description of the structural modeling task, including building geometry, building function, and seismic intensity, is denoted by D_{T} . The set of physical and engineering constraints (e.g., structural mechanics principles, design verification requirements) is represented by D_{P} . Furthermore, the target modeling APIs and the computational environment (e.g., OpenSeesPy functions, modeling workflows) are specified by D_{A} . Given this specification, the modeling target is characterized as a structural model implementation \mathcal{M}=(C,E) , where C represents the generated modeling code and E represents the execution outcomes. These outcomes encompass successful compilation, runtime stability, and physically valid responses (e.g., modal periods, internal forces). The AutoBM task is characterized as the generation of an optimal structural model \hat{M} , conditioned upon the input specification D . This optimization objective is formulated as: (1) \hat{M}=\arg\max_{M}\Pr(M\mid D) In contrast to general-purpose code-generation paradigms, the AutoBM task requires the generated code to satisfy constraints on both syntactic correctness and physical feasibility. Consequently, the optimization objective is characterized as multi-dimensional. A composite objective function \vec{F}(M) is formulated as follows: (2) \max\vec{F}(M)=\alpha_{1}S_{\mathrm{Exec}}(M\mid D_{A})+\alpha_{2}S_{\mathrm{Phys}}(M\mid D_{P})+\alpha_{3}S_{\mathrm{Spec}}(M\mid D_{T}) Here, S_{\text{Exec}}(M\mid D_{A}) evaluates executability and API compliance, while S_{\text{Phys}}(M\mid D_{P}) measures physical consistency and structural validity under mechanics principles and engineering constraints. Semantic alignment with modeling requirements is assessed by S_{\text{Spec}}(M\mid D_{T}) . The trade-offs among these objectives are regulated by non-negative weights \alpha_{i} , balancing code flexibility and physical correctness. The objective of AutoBM is therefore to generate executable, engineering-valid structural models, thereby forming a closed-loop paradigm for LLM-based building modeling. Figure 4. Overview of the CivilInstruct construction procedure. 4. BenchMark: BMBench This section introduces the modeling of building benchmark (BMBench), a novel benchmark designed for the training and evaluation of AutoBM task. The construction process, evaluation protocol, and the metrics derived from this protocol are described in detail. 4.1. CivilInstruct Dataset Construction The construction of the CivilInstruct dataset follows a four-stage pipeline, as shown in Figure 4 . First, OpenSeesPy documentation was combined with LLM-based synthesis to generate 3,800 API-learning samples and 3,100 expert-level instruction-code instances covering building functions and seismic conditions. Subsequently, 3,500 Bug-CoT samples were derived from execution error logs, and 512 high-quality code instances were filtered and annotated with structural periods obtained via finite element analysis, providing physically grounded supervisory signals. Further implementation details for each stage are provided in Appendix A.1 . 4.2. BMEval: Benchmark Design To ensure the quality, relevance, and effectiveness of the AutoBM task, BMEval was constructed through a three-stage process. First, building samples were randomly generated to cover a diverse set of attributes (e.g., building function, floor height, plan dimensions) within a reasonable structural parameter space. Then, an expert-driven mechanism was adopted to optimize and validate all design solutions in strict accordance with seismic design codes, ensuring their structural feasibility and executability. Subsequently, empirical formulas were used to compute the fundamental period of each structure as ground-truth reference values. In total, 128 labeled evaluation samples were obtained, each containing complete building information and its corresponding structural period. To evaluate LLM performance on the AutoBM task, we adopt a multidimensional evaluation protocol covering program executability, computational accuracy, and engineering compliance. For each test sample, the model independently generates n candidate programs, which are executed sequentially in a restricted sandbox. A program is deemed successful if it terminates normally within the prescribed time limit. Following the standard evaluation paradigm of mainstream code-generation benchmarks (e.g., HumanEval ( 34 ) , MBPP ( 48 ) ), we employ the unbiased Pass@k estimator to measure the probability that at least one valid solution is obtained: (3) \text{pass@k}=\begin{cases}1,&n\leq k,\\ 1-\frac{\binom{n-c}{k}}{\binom{n}{k}},&\text{otherwise}.\end{cases} Building upon Pass@k , we further define three task-specific metrics: Pass@k_{\text{period}} , Pass@k_{\text{compliance}} , and Pass@k_{\text{strict}} , corresponding to structural cycle consistency, engineering compliance, and their joint assessment. Formal definitions of these metrics are provided in Appendix A.2 . See Appendix B for the prompt used to construct the BMBench. 5. Methodology: Two-Stage Reinforcement Learning Alignment Under Structural Physical Constraints We formulate AutoBM as a conditional program synthesis problem. Given an engineering instruction x (e.g., building function, plan dimensions), a language model parameterized by \theta defines a conditional policy \pi_{\theta}(\cdot\mid x) over OpenSeesPy code sequences y=(y_{1},\ldots,y_{T}) . Our goal is to learn \pi_{\theta} that produces code that is (i) domain-compliant, (ii) logically complete, and (iii) physically valid under structural mechanics constraints. To this end, we propose RLA-SPC , a two-stage reinforcement learning framework for aligned policy optimization. As shown in Algorithm 1 , the policy is first initialized via supervised fine-tuning to ensure basic executability, and subsequently optimized using SPC-GRPO to enforce structured and physics-aware constraints. 5.1. Stage I: Domain Instruction Fine-Tuning Let \mathcal{D}=\{(x^{(n)},y^{(n)})\}_{n=1}^{N} denote the instruction dataset (CivilInstruct), where x is an AutoBM instruction and y is a reference OpenSeesPy implementation. We fine-tune a pretrained model by maximizing the conditional likelihood of the reference code: (4) \mathcal{L}_{\mathrm{SFT}}(\theta)=-\mathbb{E}_{(x,y)\sim\mathcal{D}}\left[\sum_{t=1}^{|y|}\log\pi_{\theta}(y_{t}\mid x,y_{