Applying Vision Transformers on Spectral Analysis of Astronomical Objects
Luis Felipe Strano Moraes, Ignacio Becker, Pavlos Protopapas, Guillermo Cabrera-Vives · 2025
Read Applying Vision Transformers on Spectral Analysis of Astronomical Objects by Luis Felipe Strano Moraes, Ignacio Becker, Pavlos Protopapas, Guillermo Cabrera-Vives (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00294.txt.
Introduction
Luis Felipe Strano Moraes and co-authors apply vision transformers to the spectra of stars and other objects in this 2025 paper. These models were built for image tasks, not for astronomy. The team turned one-dimensional spectra into two-dimensional plots so the models could read them.
They fine-tuned a model first trained on a large image set, using millions of spectra. The spectra came from the SDSS and LAMOST sky surveys. The model was tested on sorting objects by type and on estimating redshift. It did both tasks well and scaled to large data. The work shows image models can be reused for spectral science at survey scale.
Catalog record
| CREATOR | Luis Felipe Strano Moraes, Ignacio Becker, Pavlos Protopapas, Guillermo Cabrera-Vives |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00294
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Applying Vision Transformers on Spectral Analysis of Astronomical Objects Luis Felipe Strano Moraes, Ignacio Becker, Pavlos Protopapas, Guillermo Cabrera-Vives Abstract We apply pre-trained Vision Transformers (ViTs), originally developed for image recognition, to the analysis of astronomical spectral data. By converting traditional one-dimensional spectra into two-dimensional image representations, we enable ViTs to capture both local and global spectral features through spatial self-attention. We fine-tune a ViT pretrained on ImageNet using millions of spectra from the SDSS and LAMOST surveys, represented as spectral plots. Our model is evaluated on key tasks including stellar object classification and redshift ( z z ) estimation, where it demonstrates strong performance and scalability. We achieve classification accuracy higher than Support Vector Machines and Random Forests, and attain R 2 R^{2} values comparable to AstroCLIP’s spectrum encoder, even when generalizing across diverse object types. These results demonstrate the effectiveness of using pretrained vision models for spectroscopic data analysis. To our knowledge, this is the first application of ViTs to large-scale, which also leverages real spectroscopic data and does not rely on synthetic inputs. 1 Introduction Spectroscopy is a core observational technique in astrophysics for determining the physical and chemical properties of celestial objects ( Burrows & Orton 2010 ) . By dispersing light into a spectrum, astronomers can extract information about an object’s composition, temperature, radial motion, and even aspects of its structure or environment ( Gray 2005 ) . Unlike direct imaging, which mainly provides spatial information, spectroscopic observations probe the underlying physical processes and conditions in astronomical objects such as stars, nebulae, and galaxies. Through spectral analysis, scientists can identify the elements present in them and discern how they exist or interact under extreme cosmic conditions that cannot be replicated in laboratories ( Wahlgren 2011 ) . Spectral data are also essential for understanding the large-scale structure and evolution of the universe. The redshift of spectral lines provides a key method for measuring cosmic expansion, allowing us to estimate distances to galaxies and trace the large-scale structure of the cosmos ( Hubble 1929 ; Colless et al. 2001 ) . However, galaxy evolution is not solely dictated by their positions and motions in an expanding universe; their internal chemical composition also shapes it. Spectroscopy plays a crucial role in this aspect, revealing how elements are synthesized in stars, expelled into the interstellar medium, and recycled into subsequent generations of stars ( Maiolino & Mannucci 2019 ) . By studying absorption and emission lines, astronomers can track the abundance of elements essential for planetary formation and, ultimately, for the emergence of life ( Wolfe et al. 2005 ) . Modern spectroscopic surveys such as the Sloan Digital Sky Survey ( Kollmeier et al. 2019 ; Stoughton et al. 2002 , SDSS, ) and the Large Sky Area Multi-Object Fiber Spectroscopic Telescope ( Luo et al. 2015 , LAMOST, ) have enabled access to datasets containing millions of spectra across diverse object types. These surveys are also making continuous data releases to the public, enabling groundbreaking research to take place. The volume of spectral data continues to grow, with surveys such as ( Sheinis et al. 2023 , MSE, ) , ( de Jong et al. 2016 , 4MOST, ) , ( Gaia Collaboration et al. 2016 , GAIA, ) , and ( Hahn et al. 2023 , DESI, ) now operational or in preparation. Spectral redshift estimation and classification in surveys like SDSS typically rely on template-fitting, where observed spectra are matched to composite templates derived from empirically defined object classes ( Kügler, S. D. et al. 2015 ) . These templates are applied to each observed spectrum, allowing predefined properties such as the redshift to be computed by identifying the best fit. However, this approach simplifies the complexity of the data, limiting the precision of individual property estimates. Furthermore, the reliability of the results is sensitive to the selection and construction of the reference templates. While such automated pipelines improve efficiency over manual inspection, they still operate under constrained assumptions and leave room for more flexible, data-driven approaches that could capture finer-grained spectral features. Manual inspection introduces uncertainty in redshift determination, often estimated as \sigma_{z}/(1+z)<0.001 . One possible cause of this uncertainty, as noted by Yang et al. 2018 , is that up to 25\% of the input targets in their analysis could be categorized as unreliable due to low signal-to-noise ratios or ambiguous spectral features. So even introducing manual inspection still leaves possibilities for errors and with the size of the surveys, adding extra automated steps to help validate and cross-check means greater reliability on the results. Deep learning models offer a promising alternative by learning generalizable spectral representations from datasets. Unlike manual inspection or traditional template-based approaches, these models can capture complex patterns in spectral data and generalize across different observational conditions. Vision Transformers ( Dosovitskiy et al. 2020 , ViTs) , a specific Deep Learning architecture introduced for image recognition, are particularly promising for spectral analysis due to their ability to capture contextual information in a scalable manner that is also adaptable to multiple downstream tasks, making them suitable for a broad range of astrophysical experiments. ViTs are derived from Transformer architectures, originally developed for natural language processing ( Vaswani et al. 2017 ) , which leverage self-attention mechanisms to efficiently model long-range dependencies. Unlike recurrent architectures ( Hochreiter & Schmidhuber 1997 ) that process sequences sequentially, transformers simultaneously compute relationships between all tokens in parallel, enabling more efficient training and better scalability to long sequences and large datasets. The self-attention mechanism allows each token to assess the relevance of all other tokens, facilitating a contextual understanding that is particularly powerful in language tasks. Typically, transformers include a special classification (CLS) token, serving as a condensed representation of the entire sequence for downstream prediction tasks. When adapted for images, ViTs divide the input image into fixed-size patches, which are then linearly embedded and combined with positional encodings to preserve spatial structure. These patch embeddings pass through transformer layers, where self-attention captures global interactions across the entire image. Unlike convolutional neural networks ( LeCun et al. 1998 ; Krizhevsky et al. 2012 , CNNs) , which progressively build hierarchical features through local receptive fields, ViTs inherently model global relationships from the earliest layers. This capability motivates our approach to transform astronomical spectra into two-dimensional, image-like representations, enabling effective analysis using ViTs. ViTs typically require very large training and finetuning datasets, a requirement that modern astronomy can meet with upcoming massive surveys. We employ state-of-the-art ViTs that were pre-trained on regular images from ImageNet ( Deng et al. 2009 ) and finetune it on plots of spectra generated from a combined dataset comprising large portions of SDSS and LAMOST surveys. We evaluate these models on multiple tasks, including redshift regression, stellar parameter inference, and morphological classification. In both types of tasks (classification and regression), the model shows high accuracy and performs well across a range of signal-to-noise (SNR) ratios. Furthermore, this design allows for easy integration of more data sources, such as different surveys, and refinements to the downstream tasks being performed. These results highlight the extensibility and the potential to support a broad range of future applications with our approach. This paper is