Data-Driven Stellar Spectral Modelling with GSPICE
Douglas P. Finkbeiner, Joshua S. Speagle (沈佳士), Tanveer Karim (tanviir kirm) · 2025
Read Data-Driven Stellar Spectral Modelling with GSPICE by Douglas P. Finkbeiner, Joshua S. Speagle (沈佳士), Tanveer Karim (tanviir kirm) (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-16754.txt.
Introduction
Douglas P. Finkbeiner, Joshua S. Speagle, and Tanveer Karim present GSPICE, a data-driven method for cleaning stellar spectra without leaning on precomputed stellar templates. Spectral pipelines must mask cosmic rays, fix wavelength solutions, and estimate noise while staying neutral about what objects might appear, a tension for template-based tools.
GSPICE models an ensemble of spectra as a multivariate Gaussian and predicts each pixel's expected value and variance from the other pixels. Comparing observations against those predictions exposes outliers, which can then be masked or replaced by the estimates. Applied to 3.9 million LAMOST spectra, variants of the method caught both single-pixel cosmic ray hits and extended systematics such as bad wavelength calibrations, and characterized the true per-pixel uncertainties. An application to diffuse interstellar bands shows the value of data partitioning.
Catalog record
| CREATOR | Douglas P. Finkbeiner, Joshua S. Speagle (沈佳士), Tanveer Karim (tanviir kirm) |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-16754
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Data-Driven Stellar Spectral Modelling with GSPICE Douglas P. Finkbeiner, Joshua S. Speagle (沈佳士), Tanveer Karim (tanviir kirm) Abstract Spectral data reduction pipelines deal with a wide variety of challenges including masking cosmic rays, calibrating wavelength solutions, and estimating background noise while trying to remain model-agnostic. Traditional methods rely on hardware-specific code or pre-calculated stellar model templates to solve this problem, making them model-dependent and not suitable for large datasets that may contain new classes of objects. To solve this problem, we present a flexible, data-driven method: the GausSian PIxelwise Conditional Estimator ( GSPICE ) that models an ensemble of spectra as a multivariate Gaussian and estimates the expected value and expected variance of each pixel in each spectrum conditional on others. GSPICE compares observed fluxes and errors to its own flux and error estimates to reveal outliers, which then can be completely masked or replaced by their estimates. We apply GSPICE to 3.9 million stellar spectra from the LAMOST survey, and show that variations of the method can directly identify and correct both individual pixel-level outliers (e.g., from cosmic ray hits) as well as extended systematic features (e.g., from incorrect wavelength calibrations), while still providing a novel characterization of the true per-pixel measurement uncertainties. We also demonstrate how GSPICE can take advantage of data partitioning with an application to diffuse interstellar bands. Implementations of GSPICE in both Python and IDL can be found http://github.com/dfink/gspice . show][email protected] I Introduction Spectral data reduction is a fundamental challenge in modern astronomy. No spectroscopic dataset is perfect; observations are frequently plagued by sensor defects like bad CCD columns, transient artifacts such as cosmic-ray hits, or astrophysical interlopers like background galaxies. In fiber spectrographs, software failures can occasionally assign incorrect wavelength solutions to entire sections of a trace. Similarly, hyperspectral imaging (a 3-D dataset with a spectrum of every point in a 2-D image) suffers from complex optical distortions such as smile and keystone ( Leung et al. 2022 ) . These challenges are becoming acute in the era of massive spectroscopic surveys such as DESI ( DESI Collaboration et al. 2022 ) , Gaia ( Sartoretti et al. 2018 ) , SDSS V ( Kollmeier et al. 2017 ) , and Euclid ( Laureijs et al. 2011 ) . With millions of spectra to process, manual inspection is impossible. Traditional reduction pipelines typically rely on template matching (comparing observations to theoretical models) to identify defects. However, these methods are brittle: they only find the errors they were designed to look for (the “known unknowns”). Furthermore, when theoretical models are imperfect (as is often the case for high-redshift galaxies or complex stellar atmospheres), a valid scientific discovery might be flagged as a data error; conversely, a subtle instrument glitch might be accepted as real physics. To address this, we need a flexible way to determine if an observation conforms to expectations without relying on rigid physical models. The intuition behind this approach is straightforward and builds on the fundamental fact that stellar spectra are highly redundant, with the flux at one wavelength strongly correlated with the flux at many others due to the underlying physics of atomic transitions. This means that if we know the shape of the spectrum in some subset of wavelengths, we should be able to predict the behavior at others with high precision. If the actual data deviates significantly from this prediction, we have likely found either a processing artifact or a potential outlier. In addition to simply making a prediction, we also need to consider the relative variability of various spectral features to ensure that rare objects (which may not be represented in training data) still must be handled properly and not rejected outright. For instance, emission and absorption features from the atmosphere (e.g., sky lines) or other material along the line of sight (e.g., dust) may dominate the variance at some wavelengths, while other features might be intrinsically variable (e.g., metal-dependent absorption features). We therefore need an approach that is able to detect artifacts and repair them with accurate estimates of the mean and variance for each potential outlier pixel. Correct variance estimates are also often important for downstream analysis. In order to address both of these issues, in this work we introduce the GausSian PIxelwise Conditional Estimator ( GSPICE ). GSPICE uses the highly covariate nature of spectra to estimate the expected flux of an object at a pixel conditioned on the fluxes of other pixels. Similar to Saydjari & Finkbeiner 2022 , the Gaussian assumption of the stellar and galactic spectra enables us to use exact and closed-form solutions for pixelwise flux estimation. Furthermore, our probabilistic approach shares a theoretical lineage with recent advancements in component separation, most notably MADGICS (Marginalized Analytic Data-generated Gaussian Inference for Component Separation; Saydjari et al. 2023 ). Both GSPICE and MADGICS leverage the power of Gaussian priors to model astronomical signals. However, where MADGICS focuses on the marginalization of components to separate additive signals (e.g., separating dust emission from CMB, or tellurics from stellar spectra), GSPICE focuses on *conditional* inference at the pixel level. In contrast to approaches like principal component analysis (PCA) that compress the spectrum into a linear combination of only a modest number of “principal components” (i.e., a small number of the eigenvectors of the covariance), GSPICE is able to leverage the full covariance structure of the dataset. This not only makes the method more robust to outliers when training, but also allows for analysis of a spectrum on a pixel-by-pixel basis. By comparing the observed flux to this data-driven prediction, we can identify outliers with high sensitivity, whether they are sharp cosmic rays or broad calibration errors. Once identified, these outliers can be mathematically “in-painted” with their predicted values, effectively repairing the spectrum for downstream analysis. There are necessary limitations to this approach. To learn the correlations effectively, the spectra must be shifted to a common reference velocity (e.g., rest frame). Additionally, features that move relative to the star, such as foreground interstellar absorption, require special handling (although see Section IV ). Finally, the instrumental line-spread function at each wavelength must be approximately uniform across the sample. In Section II , we detail the probabilistic framework of the algorithm and its relationship to Gaussian Processes (GPs) and other approaches in the literature. We apply GSPICE to 3.9 million spectra from the LAMOST survey ( Luo et al. 2015 ) in Section III to demonstrate artifact detection and repair. In Section IV , we demonstrate how the method can be used to isolate foreground signals. Section V summarizes our results and future applications. Appendix A describes the mathematical and computational methodology that allows us to compute a 4000\times 4000 matrix inverse times a vector in <1 ms on a CPU core. II GausSian PIxelwise Conditional Estimation ( GSPICE ) We now describe our method in additional detail. Section II.1 gives our notation and setup. Section II.2 discusses standard approaches with principal component analysis (PCA). Section II.3 introduces the concept of a Gaussian prior on spectrum space, and contrasts our approach more explicitly with Principal Component Analysis (PCA). Section II.4 relates our discretized wavelength grid to a non-stationary Gaussian process, providing justification for interpolation of the covariance. In Section