Exoformer : Accelerating Bayesian atmospheric retrievals with transformer neural networks
L. Pagliaro {}^{\hrefhttps://orcid.org/0009-0001-7886-949X}, T. Zingales {}^{\hrefhttps://orcid.org/0000-0001-6880-5356}, G. Piotto {}^{\hrefhttps://orcid.org/0000-0002-9937-6387}, I. Giovannini {}^{\hrefhttps://orcid.org/0009-0005-9376-7044}, G. Mantovan {}^{\hrefhttps://orcid.org/0000-0002-6871-6131} · 2026
Read Exoformer : Accelerating Bayesian atmospheric retrievals with transformer neural networks by L. Pagliaro {}^{\hrefhttps://orcid.org/0009-0001-7886-949X}, T. Zingales {}^{\hrefhttps://orcid.org/0000-0001-6880-5356}, G. Piotto {}^{\hrefhttps://orcid.org/0000-0002-9937-6387}, I. Giovannini {}^{\hrefhttps://orcid.org/0009-0005-9376-7044}, G. Mantovan {}^{\hrefhttps://orcid.org/0000-0002-6871-6131} (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27623.txt.
Introduction
Pagliaro and co-authors speed up exoplanet atmosphere studies with a transformer. Bayesian retrievals on spectra can take very long. Exoformer learns to propose smart prior ranges instead. It trains for hot Jupiter transmission spectra. Tests use mock data and real JWST spectra. WASP-39b and WASP-17b serve as real cases. Retrievals run three to eight times faster. Best-fit spectra and parameters stay intact.
Catalog record
| CREATOR | L. Pagliaro {}^{\hrefhttps://orcid.org/0009-0001-7886-949X}, T. Zingales {}^{\hrefhttps://orcid.org/0000-0001-6880-5356}, G. Piotto {}^{\hrefhttps://orcid.org/0000-0002-9937-6387}, I. Giovannini {}^{\hrefhttps://orcid.org/0009-0005-9376-7044}, G. Mantovan {}^{\hrefhttps://orcid.org/0000-0002-6871-6131} |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47154 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.27623
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exoformer : Accelerating Bayesian atmospheric retrievals with transformer neural networks L. Pagliaro {}^{\hrefhttps://orcid.org/0009-0001-7886-949X}, T. Zingales {}^{\hrefhttps://orcid.org/0000-0001-6880-5356}, G. Piotto {}^{\hrefhttps://orcid.org/0000-0002-9937-6387}, I. Giovannini {}^{\hrefhttps://orcid.org/0009-0005-9376-7044}, G. Mantovan {}^{\hrefhttps://orcid.org/0000-0002-6871-6131} Abstract Computationally expensive and time-consuming Bayesian atmospheric retrievals pose a significant bottleneck for the rapid analysis of high-quality exoplanetary spectra from present and next generation space telescopes, such as JWST and Ariel. As these missions demand more complex atmospheric models to fully characterize the spectral features they uncover, they will benefit from data-driven analysis techniques such as machine and deep learning. We introduce and detail a novel approach that uses a transformer-based neural network ( Exoformer ) to rapidly generate informative prior distributions for atmospheric transmission spectra of hot Jupiters. We demonstrate the effectiveness of Exoformer using both simulated observations and real JWST data of WASP-39b and WASP-17b within the TauREx retrieval framework, leveraging the nested sampling algorithm. By replacing standard uniform priors with Exoformer -derived informative priors, our method accelerates nested-sampling retrievals by factor of 3-8 in the tested cases, while preserving the retrieved parameters and best-fit spectra. Crucially, we ensure that the retrieved parameters and the best-fit models remain consistent with results from classical methods. Furthermore, we confirm the statistical consistency of the two retrieval approaches by comparing their log-Bayesian evidence, obtaining absolute values of each Bayes factor | Δ log Z | < 5 |\Delta\log{Z}|<5 , i.e., with no strong preference following common scales for either model. This hybrid approach significantly enhances the efficiency of atmospheric retrieval tools without compromising their accuracy, paving the way for more rapid analysis of complex exoplanetary spectra and enabling the integration of more realistic atmospheric models. 1 Introduction The advent of the James Webb Space Telescope (JWST) ( 15 ) and forthcoming Ariel ( 47 ) space missions opens the door to unprecedented spectroscopic observations of exoplanetary atmospheres. The study of atmospheric compositions plays a crucial role in understanding how planets form and evolve, as well as in detecting molecular signatures of life. Many tools based on Bayesian statistics have been developed to retrieve molecular abundances, temperatures, and many other atmospheric parameters from spectroscopic observations (e.g., TauREx ( 2 ) , NEMESIS ( 21 ) , and petitRADTRANS ( 34 ) ; a more complete list can be found in 31 ). The classical approach involves a Bayesian framework to obtain the posterior distributions of atmospheric parameters given an observed spectrum. These tools prove to be effective with low-resolution observations of the HST and Spitzer telescopes. However, high-quality data from JWST and Ariel show a completely new set of spectral features that can be described only with more complex atmospheric models ( 40 ) , considering for example atmospheres as multidimensional structures where physical phenomena such as convection and chemical disequilibrium occur. The resulting increase in parameters describing the atmospheric model yields a strong computational bottleneck and challenges in achieving convergence to a solution. In recent years, an increasing number of machine learning and deep learning algorithms have been implemented in the exoplanetary field, ranging from transit detection in light curves (e.g., 33 , 42 ) to atmospheric characterization (e.g., 54 , 19 , 52 ). These algorithms are based on a data-driven approach, which means that they can automatically learn to solve a given task by iteratively extracting multiple features from a large dataset ( 22 ) . Machine learning encompasses a broad range of algorithms derived from different learning paradigms (e.g., decision trees ( 39 ) , support vector machines ( 9 ) , and clustering ( 24 ) ). Deep learning by contrast relies on the artificial neuron – a mathematical function that employs nonlinear transformations through weighted inputs to process and learn from data – and its variants (e.g., convolution and attention). Individual neurons are typically grouped into distinct layers that are then repeated and connected to form more complex architectures, called deep neural networks. Deep neural networks are used extensively in complex tasks (such as images, texts, and sequential data processing), as they outperform machine learning in extracting hidden features and patterns from domains with large, high-dimensional data ( 25 ; 22 ) . Bayesian retrievals require millions of atmospheric forward models per single observation, becoming extremely slow and computationally intensive when complex atmospheric models with a large number of parameters are taken into account. Therefore, with the thousands of spectra expected from new telescopes, the efficiency of repeated analysis will be significantly impacted. The integration of deep learning emerges as a promising solution to address the computational challenges of high-dimensional Bayesian atmospheric retrievals. There are multiple pathways through which we can achieve this goal: replacing the Bayesian tool (e.g., 58 ) to directly provide parameter posterior distributions, substituting the radiative transfer code with a surrogate model (e.g., 19 ), or generating informative priors (e.g., 17 ) in place of uniform priors. In our work, we adopted a state-of-the-art deep-learning architecture, the transformer, whose architecture was proposed by 53 in the context of human language modeling to overcome the limitations of convolutional neural networks (CNN; ( 26 ) ) and long-short term memory (LSTM; 20 ) networks. Building on the success of transformers in predicting stellar parameters – as demonstrated in recent studies (e.g., 35 , 56 ) – we propose a transformer-based approach for analyzing spectroscopic data of exoplanetary atmospheres. Just as stellar spectra exhibit interconnected emission and absorption features across the spectral domain, a transformer architecture is well suited to capture these complex relationships within exoplanetary atmospheric data. We developed a tool that can retrieve the approximated posterior distributions of six atmospheric parameters using the Monte Carlo (MC) dropout technique ( 14 ) . In Section 2 we describe the transformer architecture and its fundamental operation, self-attention. In Section 3 we introduce Exoformer , our transformer-based tool, and describe its structure. In Section 4 we describe the training process of Exoformer and how uncertainties are estimated in the neural network. In Sections 5 and 6 , we show the results of applying Exoformer to simulated and real JWST transmission spectra. 2 Transformer neural networks Transformers are a type of neural network designed to learn useful representations of sequential data through a mechanism called self-attention. In fact, unlike other architectures such as CNN and LSTM, transformers have the ability to capture long-range dependencies and correlations in sequences. In the following paragraphs, we describe the most important components of a transformer algorithm. 2.1 Self-attention mechanism Self-attention is the core mechanism of the transformer architecture: it allows the model to relate each element of a sequence to all the other elements of the same sequence. In general, the i -th output \mathbf{a}_{i}\in\mathbb{R}^{D} (where D is the embedding dimension) of the self-attention operation is a weighted sum of the N inputs \mathbf{x}_{1},\dots,\mathbf{x}_{N} , with \mathbf{x}_{j}\in\mathbb{R}^{D} : \mathbf{a}_{i}=\sum_{j=1}^{N}{A}_{ij}\mathbf{x}_{j}, (1) where \mathbf{A}\in\mathbb{R}^{N\times N} is the attention matrix, whose elements are normalized between [0,1] and their sum equals to 1 . 53 applied the self-attention mechanism to deep learning by introducing a set of learnable matrices to compute the attention matrix and the attention output. Given the sequence of N inputs \mathbf{x}_{1},\dots,\mathbf{x}_{N}\in\mathbb{R}^{D} , each vector is first linearly transformed by three distinct learnable matrices \mathbf{W}\in\mathbb{R}^{D\times D} : \displaystyle\mathbf{q}_{i}=\mathbf{x}_{i}\mathbf{W}_{Q}, (2) \displaystyle\mathbf{k}_{i}=\mathbf{x}_{i}\mathbf{W}_{K}, (3) \displaystyle\mathbf{v}_{i}=\mathbf{x}_{i}\mathbf{W}_{V}. (4) The