Scalable heliostat surface predictions from focal spots: Sim-to-Real transfer of inverse Deep Learning Raytracing
Jan Lewen, Max Pargmann, Mehdi Cherti, Jenia Jitsev, Robert Pitz-Paal, Daniel Maldonado Quinto · 2025
Read Scalable heliostat surface predictions from focal spots: Sim-to-Real transfer of inverse Deep Learning Raytracing by Jan Lewen, Max Pargmann, Mehdi Cherti, Jenia Jitsev, Robert Pitz-Paal, Daniel Maldonado Quinto (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03712.txt.
Introduction
Jan Lewen and co-authors show that a method for reading solar mirror flaws from their light spots now works on real hardware, in a 2025 paper. Solar power towers focus sunlight with fields of mirrors called heliostats. Tiny surface flaws scatter each mirror's spot and cut the plant's output, yet measuring every mirror's surface directly is impractical. Earlier work proposed inferring the surface from spot images taken during normal calibration, using deep learning ray tracing in reverse.
This study carries that method from simulation to real data for the first time. The transfer matters because control systems today assume perfect mirror surfaces, which costs efficiency and can create safety risks. Accurate surface maps from routine images could sharpen plant control at scale.
Catalog record
| CREATOR | Jan Lewen, Max Pargmann, Mehdi Cherti, Jenia Jitsev, Robert Pitz-Paal, Daniel Maldonado Quinto |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-03712
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Scalable heliostat surface predictions from focal spots: Sim-to-Real transfer of inverse Deep Learning Raytracing Jan Lewen, Max Pargmann, Mehdi Cherti, Jenia Jitsev, Robert Pitz-Paal, Daniel Maldonado Quinto Abstract Concentrating Solar Power (CSP) plants are a key technology in the transition toward sustainable energy. A critical factor for their safe and efficient operation is the accurate distribution of concentrated solar flux on the receiver. However, flux distributions from individual heliostats are highly sensitive to surface imperfections, such as canting and mirror deformations. Measuring these surfaces across hundreds or thousands of heliostats remains impractical in real-world deployments. As a result, control systems often assume idealized heliostat surfaces, leading to suboptimal performance and potential safety risks. To address this, inverse Deep Learning Raytracing (iDLR) [ 1 ] has been introduced as a novel method for inferring heliostat surface profiles from target images recorded during standard calibration procedures. In this work, we present the first successful Sim-to-Real transfer of iDLR, enabling accurate surface predictions directly from real-world target images. We evaluate our method on 63 heliostats under real operational conditions. iDLR surface predictions achieve a median mean absolute error (MAE) of 0.17 mm and show good agreement with deflectometry ground truth in 84% of cases. When used in raytracing simulations, it enables flux density predictions with a mean accuracy of 90% compared to deflectometry over our dataset, and outperforms the commonly used ideal heliostat surface assumption by 26 %. We tested this approach in a challenging double-extrapolation scenario—involving unseen sun positions and receiver projections—and found that iDLR maintains high predictive accuracy, highlighting its generalization capabilities. Our results demonstrate that iDLR is a scalable, automated, and cost-effective solution for integrating realistic heliostat surface models into digital twins. This opens the door to improved flux control, more precise performance modeling, and ultimately, enhanced efficiency and safety in future CSP plants. List of Abbreviations CSP Concentrating Solar Power STJ Solar Tower Jülich DLR Deutsches Zentrum für Luft- und Raumfahrt iDLR inverse Deep Learning Raytracing GAN Generative Adversarial Network GPU Graphics Processing Unit NURBS Non-Uniform Rational B-Spline MAE Mean Absolute Error SSIM Structural Similarity Index CSR Circumsolar Ratio 1 Introduction Concentrated Solar Power (CSP) is a key technology for the decarbonization of energy systems, offering the capability to deliver dispatchable electrical energy [ 2 ] and to drive solar thermochemical reactions for fuel production [ 3 ] . A crucial aspect of efficient CSP plant operation is the flux density distribution on the receiver, as each receiver has specific thermal thresholds. Proper flux distribution minimizes material stress and degradation, thereby optimizing receiver performance. While the effects of tracking errors and the hence mandatory automatized calibration have been extensively studied [ 4 , 5 , 6 , 7 , 8 , 9 ] , recently obtaining accurate flux density predictions for individual heliostats from easily available data sources for further optimized receiver performance has become a focus of research [ 10 , 11 , 12 , 1 ] . Traditionally, highly accurate flux density predictions required reconstructing the heliostat surface, accounting for mirror and canting errors, and simulating the resulting flux using raytracing techniques [ 13 , 14 ] . Established methods for measuring heliostat surface deformation include deflectometry [ 13 ] , laser scanning [ 15 , 16 ] , photogrammetry [ 17 , 18 , 19 , 20 , 21 ] , and flux mapping [ 22 ] (see Arancibia-Bulnes et al. [23] for a review). Despite their accuracy, these methods require manual labor and additional hardware, making them expensive and impractical for commercial-scale deployment aimed at creating digital twins of heliostats for better heliostat control. Hence, in most cases flux density predictions for heliostat control are made either through raytracing of an ideal heliostat or by adopting simplified flux density assumptions. To overcome this modelling error undermining the safe and efficient operation of the power plant, several works were done to obtain very precise flux density predictions without the need of new hardware installation and manual work. Zhu et al. [10] introduced a post-installation calibration procedure that estimates four geometric parameters per facet through optimization, using only target images of the heliostat’s focal spot. Despite the simplicity of the heliostat model—assuming only canting and focusing errors—the approach demonstrated significant efficiency gains up to 20%. This highlights the potential of integrating accurate flux density prediction into aim point optimization strategies. However, the model used by Zhu et al. [10] is limited in scope. For example, the heliostats at Solar Tower Jülich (STJ) in Germany have 4 canted facets, however the individually facets are flat and non-focussing. Their flux density is significantly influenced by surface characteristics such as waviness and facet edge bending, which are not captured by their method. To address this, Pargmann et al. [11] proposed a more generic heliostat model based on a differentiable raytracing framework. Their method optimizes a NURBS spline to match target images, allowing for more accurate flux density prediction. Nevertheless, due to the ill-posed and underdetermined nature of the inverse problem, the surface prediction is in most cases poor. This introduces uncertainty when extrapolating flux density from the lambertian target plane to the receiver plane with potentially complex geometry. Alternatively, Kuhl et al. [12] presented a fully data-driven approach, representing each heliostat by a latent vector with no physical interpretation. This method achieves highly accurate flux density predictions and is straightforward to implement, as it requires no physical modeling. However, the lack of interpretability limits its integration with other physically grounded factors affecting flux density, such as soiling models, sun shape effects, shading, blocking, or extrapolation to complex receivers. To overcome the limitations of the above mentioned approaches, Lewen et al. [1] recently introduced inverse Deep Learning Raytracing (iDLR), a novel deep learning approach that infers heliostat surface information from target images of focal spots. This method leverages the shape of the flux distribution to predict the underlying surface characteristics. iDLR has demonstrated high accuracy and strong agreement with state-of-the-art surface measurement techniques such as deflectometry, based on simulated flux densities. However, up to date, its application was only shown on synthetic data, and the model has not yet been transfered to real-world measurements. Recent advancements in deep learning have been fueled by the availability of large-scale datasets. However, across domains such as robotics [ 24 , 25 , 26 ] , autonomous driving [ 27 , 28 ] , and drone navigation [ 29 ] or as mentioned above iDLR [ 1 ] exclusive reliance on real-world data for training is often infeasible due to the high costs and time required for data acquisition. Consequently, simulated data play a central role in training models that are ultimately deployed in real-world settings. However, the inherent domain discrepancy between synthetic and real data—commonly referred to as the sim-to-real gap [ 24 ] —poses a significant challenge to direct transferability, often leading to poor generalization in real-world environments. To enable a robust Sim-to-Real transfer of iDLR, three general strategies are conceivable. The first is fine-tuning the model on a limited amount of real-world data [ 30 ] . The second involves Domain Adaptation , where the discrepancy between the simulated and target domains is bridged, often using unsupervised deep learning models such as