Toward a foundational thermal model for residential buildings DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 22–25, 2026; Banff, Alberta, Canada ISBN: 978-1-4503-XXXX-X/2018/06

Ting-Yu Dai, Kingsley Nweye, Dev Niyogi, Zoltan Nagy · 2026

Read Toward a foundational thermal model for residential buildings DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 22–25, 2026; Banff, Alberta, Canada ISBN: 978-1-4503-XXXX-X/2018/06 by Ting-Yu Dai, Kingsley Nweye, Dev Niyogi, Zoltan Nagy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01364.txt.

Introduction

Ting-Yu Dai, Kingsley Nweye, Dev Niyogi, and Zoltan Nagy take a step toward a single pre-trained model that can predict indoor heat flow for many homes at once. Today each building tends to need its own tuned model, which slows smart heating and cooling control.

Their design is a physics-informed Transformer that bakes in heat rules, such as rate-of-change features and step-by-step numerical updates, inside a decoder-only frame. It also reads fixed traits of each home from its simulation file. On a set of 247 homes in three climate zones, one-step forecasts land within about three tenths of a degree. A model trained on as few as two homes still works on new homes and new climates with no extra tuning.

Catalog record

CREATORTing-Yu Dai, Kingsley Nweye, Dev Niyogi, Zoltan Nagy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.01364

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Toward a foundational thermal model for residential buildings DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 22–25, 2026; Banff, Alberta, Canada ISBN: 978-1-4503-XXXX-X/2018/06 Ting-Yu Dai, Kingsley Nweye, Dev Niyogi, Zoltan Nagy Abstract . The building energy community lacks a foundational thermal model, i.e., a single pretrained model capable of generalizing across diverse buildings, climates, and control strategies without building-specific calibration. Achieving this vision requires architectural principles that capture universal thermal dynamics rather than memorizing building-specific patterns. We take a step toward this goal by presenting a physics-informed transformer architecture that embeds domain knowledge, e.g., derivative enrichment and Euler-based numerical integration, into a decoder-only framework. We incorporate static building features extracted from simulation models and employ Rotary Position Embedding attention to capture temporal dependencies. Evaluated on the CityLearn dataset spanning 247 residential buildings across three climate zones, our model achieves one-step prediction accuracy (RMSE of 0.30°C in Texas, 0.29°C in Vermont) while outperforming both traditional baselines and fine-tuned Time-Series Foundation Models. We also demonstrate zero-shot transferability: models trained on as few as two buildings generalize to unseen buildings and climate zones without fine-tuning. Despite the limitation of simulated residential buildings, our results establish physics-informed architectural principles as a promising foundation for universal building thermal models. 1. Introduction Figure 1 . Physics Transformer architecture. Past observations and their derivatives are encoded by RoPE attention layers, then enriched with static building features ( H_{S} ). A second attention stack integrates future-known covariates ( H_{k} ) before predicting the temperature change \Delta T_{t} The inset (right) shows the transformer block structure with RoPE applied to queries and keys. Buildings account for 40% of global energy consumption, making them central to decarbonization efforts ( 19 ) . Their inherent thermal storage and controllable loads enable demand flexibility, but realizing this potential requires accurate thermal models for advanced control strategies such as Model Predictive Control (MPC) and Reinforcement Learning (RL). The successful realization of this potential, however, is essentially predicated on the deployment of robust and adaptive control strategies, Model Predictive Control (MPC) and Reinforcement Learning (RL). The efficacy of these advanced controllers is highly correlated to the availability of accurate, reliable, and computationally tractable building energy models (BEMs) that capture the thermal dynamics of buildings. Without a precise understanding and prediction of a buildings’ thermal behavior, optimizing energy operations becomes an intractable challenge, enabling faster ways to simulate thermal dynamics, potentially through data-driven approaches. Despite significant research, developing control-oriented thermal models encounters persistent hurdles. Physics-based models require building-specific calibration; data-driven models need large datasets and generalize poorly; grey-box RC models struggle with nonlinearities. Recent Time-Series Foundation Models (TSFMs) exhibit limited generalizability for building energy tasks ( 12 ; 14 ) , and their scale is mismatched with control latency requirements. Prior work on LSTM-based models ( 6 ; 5 ) and few-shot transfer ( 18 ; 10 ) demonstrates potential but lacks unified standards for cross-building generalization. Thus, our driving research question is: how can we build a transferable thermal model capable of zero-shot generalization to unseen buildings? The success of foundation models in natural language processing and computer vision has sparked interest in developing analogous models for engineering domains ( 3 ; 20 ) . A foundational thermal model for buildings would ideally exhibit three properties: (1) zero-shot generalization to unseen buildings without fine-tuning, (2) cross-domain transfer across building types, climates, and operational contexts, and (3) sample efficiency requiring minimal data from new buildings for adaptation. Achieving these properties for buildings presents unique challenges absent in language or vision domains. Buildings are governed by well-understood physics (heat transfer, thermodynamics), but exhibit heterogeneity in geometry, materials, systems, and occupancy. Unlike images or text, building time series are inherently multi-modal, coupling weather, occupant behavior, and HVAC operation. Also, obtaining labeled training data requires either expensive real-world instrumentation or computationally intensive physics-based simulation. We hypothesize that foundational thermal models require physics-informed architectural priors, not only large amounts of data. While general Time-Series Foundation Models (TSFMs) achieve broad coverage through massive pre-training, they lack inductive biases aligned with physics. We test this hypothesis by integrating Euler-based residual prediction (learning temperature changes rather than absolute values) and derivative feature augmentation (explicitly encoding rate-of-change dynamics) into a decoder-only transformer with Rotary Position Embedding (RoPE) attention. We do not claim to have achieved a complete foundational model; rather, we establish architectural principles and provide empirical evidence for zero-shot transfer that we believe are necessary steps toward this goal. Our specific contributions are: • Physics-Informed Architectural Principles: We identify and validate two inductive biases, i.e., derivative enrichment and Euler-based residual prediction, that embed heat transfer physics into neural architectures. We demonstrate that these principles enable data-efficient transfer learning. • Empirical Evidence for Zero-Shot Transfer: We provide the first systematic evaluation of thermal model transferability across both buildings (within-climate) and climate zones (cross-climate). • Benchmarking the Foundation Model Paradigm: We compare physics-informed design against the prevailing foundation model approach (scale + generic architecture). The next Section states the problem and the introduces our proposed Physics Transformer architecture. Then, Section 3 details the experimental setup. Section 4 presents the results and discusses their implications, while Section 6 concludes the paper. 2. Methodology 2.1. Problem Statement Our goal is to build a building thermal model that forecasts the indoor temperature T at target timestamp t of building j : (1) T_{t}=f\left(T_{t-n:t-1},W_{t-n:t-1},S_{j},K_{t}\right) The model receives the past n time points as a time series, which contains past target observation T_{t-n:t-1} and other past-observed covariates W_{t-n:t-1} , static covariates S from building j , and future-known covariates K_{t} . The key challenge is learning thermal dynamics that generalize to unseen buildings and climates. 2.2. Proposed Physics Transformer Architecture Figure 1 illustrates the proposed architecture, which combines a decoder-only transformer backbone ( 3 ) with static feature enrichment ( 9 ) . Past observations and their temporal derivatives are embedded into a 64-dimensional hidden space, then encoded by N RoPE attention layers to produce a temporal representation ( H_{t-1} ). Static building features ( H_{S} ) enrich this representation before a second attention stack integrates future-known covariates ( H_{k} ). The output layer predicts the temperature change \Delta T_{t} rather than the absolute temperature. We now describe each component: Single Step Prediction Our model predicts only the next timestep temperature, following the autoregressive paradigm where each prediction conditions on ground-truth history. While multi-horizon output heads can improve forecasting efficiency ( 3 ) , they require assuming known future covariates (e.g., HVAC loads), which is typically unavailable in realistic control scenarios where the controller determines future actions. Decoder-only Architecture We adopt a decoder-only transformer architecture, which is inherently autoregressive: each prediction depends on the preceding sequence. This aligns naturally with thermal forecasting, where the next temperature depends on historical states. The architecture has proven effective for time series foundation models ( 3 ) . Static Enrichment The difference in thermal mass, insulation, and geometry of buildings fundamentally shapes their thermal response. We incorporate these time-invariant characteristics through static enrichment, inspired by the Temporal Fusion Transformer ( 9 ) . Unlike TFT’s repeated static injection for interpretability, we apply a single additive fusion post-temporal encoding for computational efficiency. Specifically, static building

Read the complete text