NEURIPS2024 Ariel Data Challenge: Characterization of Exoplanetary Atmospheres Using a Data-Centric Approach
Jérémie BLANCHARD, Lisa CASINO, Jordan GIERSCHENDORF · 2025
Read NEURIPS2024 Ariel Data Challenge: Characterization of Exoplanetary Atmospheres Using a Data-Centric Approach by Jérémie BLANCHARD, Lisa CASINO, Jordan GIERSCHENDORF (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08940.txt.
Introduction
A data challenge run with a major computing conference asked teams to read the air of distant planets from simulated light data. Jeremie Blanchard, Lisa Casino, and Jordan Gierschendorf report in 2025 how their team tackled the task with a data-first approach. Rather than tuning hard for the contest, they aimed for methods that would generalize to new data.
They tried several routes: shaping features, transforming signals, and modeling how uncertainty varies across readings. Good uncertainty estimates mattered a lot, moving their score by several points, and their overall score rose by eleven percent. Still, they found plain table-based modeling had clear limits for this task. Their report weighs the trade-offs between simple, clear models and raw contest performance.
Catalog record
| CREATOR | Jérémie BLANCHARD, Lisa CASINO, Jordan GIERSCHENDORF |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-08940
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
NEURIPS2024 Ariel Data Challenge: Characterization of Exoplanetary Atmospheres Using a Data-Centric Approach Jérémie BLANCHARD, Lisa CASINO, Jordan GIERSCHENDORF Abstract The characterization of exoplanetary atmospheres through spectral analysis is a complex challenge. The NeurIPS 2024 Ariel Data Challenge [ 1 ] , in collaboration with the European Space Agency’s (ESA) Ariel mission [ 2 ] , provided an opportunity to explore machine learning techniques for extracting atmospheric compositions from simulated spectral data. In this work, we focus on a data-centric business approach, prioritizing generalization over competition-specific optimization. We briefly outline multiple experimental axes, including feature extraction, signal transformation, and heteroskedastic uncertainty modeling. Our experiments demonstrate that uncertainty estimation plays a crucial role in the Gaussian Log-Likelihood (GLL) score, impacting performance by several percentage points. Despite improving the GLL score by 11%, our results highlight the inherent limitations of tabular modeling and feature engineering for this task, as well as the constraints of a business-driven approach within a Kaggle-style competition framework. Our findings emphasize the trade-offs between model simplicity, interpretability, and generalization in astrophysical data analysis. 1 Introduction The characterization of exoplanetary atmospheres is a fundamental challenge in modern astrophysics. The ability to extract precise chemical compositions from observed spectra is crucial to understanding planetary formation, evolution, and the potential habitability of distant worlds. The NeurIPS 2024 Ariel Data Challenge, organized in collaboration with the European Space Agency’s Ariel mission, presents a unique opportunity to explore data-centric methodologies [ 3 , 4 , 5 ] for analyzing exoplanetary atmospheres. This competition challenges researchers to develop algorithms capable of extracting meaningful atmospheric compositions from simulated spectral observations, which are inherently noisy and constrained by real-world instrumental limitations. Drawing from our experience as an applied research centre, our team implemented a data-centric approach to address this challenge. Our business-oriented methodology adheres to best practices, aiming to achieve consistent performance across both private and public datasets, even when the test set is unknown and inherently out of distribution. This approach led to the development of a robust and generalizable solution, avoiding any influence from reverse engineering of the process used to generate the data. We focus on feature engineering and data quality enhancement to improve the reliability of machine learning models. To streamline our submission process, we developed a pipeline that enabled rapid iteration across different approaches. This paper presents our methodological framework, highlighting key insights from our feature engineering efforts and data refinement strategies. Although our solution did not rank among the top-performing models in the competition, our work offers valuable perspectives on the impact of data quality and pre-processing on model generalization. We also discuss the challenges encountered and the trade-offs between complex model architectures and well-prepared data. Through our participation in the NeurIPS 2024 Ariel Data Challenge, our aim is to highlight the importance of data-centric approaches [ 3 , 4 , 5 ] in machine learning for exoplanetary atmospheric characterization. Data-centric AI is an approach that emphasizes improving the quality, consistency, and labeling of data rather than focusing solely on model architecture. It prioritizes systematic data curation and iteration to enhance model performance, particularly in high-variance real-world applications. The insights gained from this competition could inform future studies on optimizing retrieval methods by prioritizing data quality over model complexity. 2 Challenge Description and Data 2.1 Challenge Objectives One of the key challenges in exoplanetary research is the extraction of atmospheric spectra from raw observational data, a critical step in enabling the chemical characterization of distant worlds. In this competition, synthetic data are collected by simulated instruments on board spacecraft designed for space-based observations, such as ESA telescopes. During transit spectroscopy, a small fraction of starlight filters through the atmosphere of a planet before being captured by these instruments, carrying valuable spectral signatures of its composition. However, these signals are often buried in instrument noise, with jitter noise—caused by spacecraft microvibrations—being a major limiting factor [ 2 ] . This effect, comparable in magnitude to the planetary signal itself, complicates the accurate retrieval of spectral information, particularly for small exoplanets. Participants in this competition must face the challenge of detrending sequential 2D images of the spectral focal plane, recorded over extended observational periods, to extract the planet’s transmission spectrum (283 wavelengths) along with an uncertainty estimate. This process requires advanced noise filtering and signal reconstruction techniques to mitigate instrument-related distortions while preserving the integrity of atmospheric features. Developing robust approaches to this task is essential for ensuring that future space missions, including Ariel, can achieve their ambitious scientific goals. The competition uses the Gaussian Log-Likelihood (GLL) score, which evaluates spectral extraction by considering both accuracy and uncertainty estimation. Unlike traditional metrics, GLL penalizes overconfident predictions, ensuring that models provide realistic confidence intervals alongside spectral reconstructions. The GLL score function is defined as: GLL(y,\mu_{user},\sigma_{user})=-\frac{1}{2}\left(\log(2\pi)+\log(\sigma_{user}^{2})+\frac{(y-\mu_{user})^{2}}{\sigma_{user}^{2}}\right) (1) where y is the ground truth pixel level spectrum, \mu_{user} is the predicted spectrum, and \sigma_{user} is the corresponding uncertainty. The GLL values are summed across all wavelengths and the entire test set to produce a final GLL value ( L in equation ( 2 )), which is then transformed into a score using the following conversion function: \text{score}=\frac{L-L_{ref}}{L_{ideal}-L_{ref}} (2) where L_{ideal} is defined as the case where the submission perfectly matches the ground truth values, with an uncertainty of 10 parts per million (ppm), and L_{ref} is defined using the mean and variance of the training dataset as its prediction for all instances. This metric impacts data processing by requiring: • Balancing accuracy and uncertainty to avoid penalization. • Preserving spectral features while mitigating noise. • Ensuring robust error estimation for reliable atmospheric characterization. By enforcing these principles, the GLL score aligns with real-world challenges in exoplanet spectroscopy, where both signal integrity and uncertainty quantification are crucial. More information on the score is discussed in Section 5.1.6 . 2.2 Data The Ariel Data Science Competition provided image-based datasets, which we transformed into multiple time series to better capture planetary transits, as illustrated in Figure 1 . This conversion was the first step in our data exploration phase and allowed us to precisely identify when an exoplanet passed in front of its star, enabling us to isolate the most relevant data segments and maximize the impact of our model. Finally, our predictions were made in the spectral domain to accurately identify the chemical composition of exoplanetary atmospheres. Figure 1: Illustration of the transformation applied to the raw images signal, from image domain to time domain, in order to predict the spectral domain. . The Ariel mission includes two instruments for exoplanet observation: • FGS1 (Fine Guidance System - Channel 1): A visible-light photometer ( 0.60 – 0.80\,\mu m ) primarily designed for spacecraft centering and guiding, but also providing high-precision photometric measurements of the target star. This system is responsible for flux slope variations that affected some spectral measurements (more details in Section 5.1 ). • AIRS-CH0 (Ariel InfraRed Spectrometer - Channel 0): An infrared spectrometer 1.95 – 3.90\,\mu m , with a resolving power R\approx 100 , that captures detailed atmospheric spectral features, crucial for molecular identification in exoplanetary atmospheres. The dataset consisted of 2D images from exoplanets orbiting different types of stars, each with varying physical and spectral characteristics. However, the training dataset did not contain