Combining datasets with different ground truths using Low-Rank Adaptation to generalize image-based CNN models for photometric redshift prediction

Vikram Seenivasan [email protected], Srinath Saikrishnan [email protected], Andrew Lizarraga [email protected], Jonathan Soriano [email protected], Bernie Boscoe [email protected], Tuan Do [email protected] · 2026

Read Combining datasets with different ground truths using Low-Rank Adaptation to generalize image-based CNN models for photometric redshift prediction by Vikram Seenivasan [email protected], Srinath Saikrishnan [email protected], Andrew Lizarraga [email protected], Jonathan Soriano [email protected], Bernie Boscoe [email protected], Tuan Do [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-00146.txt.

Introduction

Vikram Seenivasan leads five coauthors in merging two kinds of galaxy data to estimate cosmic distances. Their 2026 study predicts photometric redshifts from galaxy images. One dataset is broad but less accurate. The other is precise but covers only bright galaxies. Precise redshifts also take far longer to collect. The team asks whether a small adapter can join both strengths.

They borrow Low Rank Adaptation from language model practice. A base network trains on the broad dataset first. Small adapter layers then tune it on the precise set. The adapted model shows far less bias and scatter than plain transfer learning. Full retraining on both datasets does better still. But it costs much more computing time. The authors present adapter tuning as a middle path. It may help reuse pretrained models in data poor astronomy tasks.

Catalog record

CREATORVikram Seenivasan [email protected], Srinath Saikrishnan [email protected], Andrew Lizarraga [email protected], Jonathan Soriano [email protected], Bernie Boscoe [email protected], Tuan Do [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-00146

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Combining datasets with different ground truths using Low-Rank Adaptation to generalize image-based CNN models for photometric redshift prediction Vikram Seenivasan [email protected], Srinath Saikrishnan [email protected], Andrew Lizarraga [email protected], Jonathan Soriano [email protected], Bernie Boscoe [email protected], Tuan Do [email protected] Abstract In this work, we demonstrate how Low-Rank Adaptation (LoRA) can be used to combine different galaxy imaging datasets to improve redshift estimation with CNN models for cosmology. LoRA is an established technique for large language models that adds adapter networks to adjust model weights and biases to efficiently fine-tune large base models without retraining. We train a base model using a photometric redshift ground truth dataset, which contains broad galaxy types but is less accurate. We then fine-tune using LoRA on a spectroscopic redshift ground truth dataset. These redshifts are more accurate but limited to bright galaxies and take orders of magnitude more time to obtain, so are less available for large surveys. Ideally, the combination of the two datasets would yield more accurate models that generalize well. The LoRA model performs better than a traditional transfer learning method, with ∼ 2.5 × \sim 2.5\times less bias and ∼ \sim 2.2 × \times less scatter. Retraining the model on a combined dataset yields a model that generalizes better than LoRA but at a cost of greater computation time. Our work shows that LoRA is useful for fine-tuning regression models in astrophysics by providing a middle ground between full retraining and no retraining. LoRA shows potential in allowing us to leverage existing pretrained astrophysical models, especially for data sparse tasks. 1 Department of Physics and Astronomy, UCLA, Los Angeles, CA 90095 2 Department of Computer Science, UCLA, Los Angeles, CA 90095 3 Department of Statistics and Data Science, UCLA, Los Angeles, CA 90095 4 Department of Computer Science, Southern Oregon University, Ashland, OR, 97520 1 Introduction Astronomy is in the era of Big Data, with several current and upcoming surveys such as LSST [ 13 ] and Euclid [ 17 ] imaging billions of galaxies to constrain cosmological models governing the origin and evolution of the universe. Galaxy redshifts are used to determine distances to galaxies, allowing us to map the structure of the universe through cosmic time, to determine properties of dark energy and dark matter that govern the accelerated expansion of the universe. Astrophysicists are increasingly adopting machine learning techniques to efficiently make measurements, such as galaxy redshifts, from large datasets [ 26 , 20 ] . Such measurements often require leveraging different sources of ground truth. In this paper, we explore using Low-Rank adaptation (LoRA) [ 12 ] and traditional transfer learning to fine-tune redshift prediction models for galaxies by integrating datasets with different sources of ground truth. 1.1 Related Works Redshift estimation: Traditionally, redshifts are obtained using spectroscopy. These are highly accurate but limited to bright galaxies that have strong emission lines, and are not available at the scale of large upcoming surveys as they are expensive [ 20 ] . Photometric redshifts are derived from galaxy photometry (brightness) in different wavelength bands to sample spectral properties. These are less accurate [ 2 ] but enable the analysis of much larger datasets. Recently, many machine learning models have been proposed to estimate photometric redshifts [ 8 , 6 , 4 , 2 , 25 , 14 , 15 , 26 , 27 ] . For example, convolution neural networks (CNNs) produce some of the most precise redshift estimates [ 21 , 27 , 9 , 16 ] by using galaxy images directly. These methods usually rely on spectroscopic redshift as ground truth for training. Recent work by [ 5 , 29 ] explored methods of incorporating photometric redshifts from multi-band photometry as a different source of ground truth. However, these methods are trained on photometry alone, losing out on the wealth of information in images [ 18 ] . LoRA and fine-tuning: LoRA is an established technique for fine-tuning large language models that adds adapter networks with lower-rank weight matrices to pre-trained models. This allows efficient fine-tuning of large base models, with only a tiny fraction of the parameters being updated [ 12 ] . Low-Rank adaptation has been used in astrophysics to fine-tune pre-trained models from other domains for astrophysics tasks. For example, LLMs for token-based redshift estimation [ 23 ] , foundation models for radio astronomy [ 24 ] , and diffusion models for galaxy image generation [ 3 ] . Its utility for regression-based tasks is not fully explored in astrophysics. 1.2 Contribution We explore using LoRA and transfer learning to integrate different sources of ground truth for a redshift regression CNN that takes 5-band galaxy images as input. We adapt the recently developed LoRA implementations [ 19 ] for CNNs for astrophysical data. To our knowledge, this work is the first application of LoRA for regression-based tasks for astrophysics. Previous work in astrophysics used pre-trained models from other domains—this is the first work that uses baseline models trained on astronomical data. These are key considerations vital for large-scale surveys that will leverage previous astrophysics knowledge. 2 Data In this work we use three datasets, one for the base model, one for transfer learning, and a combination dataset of the two. The dataset used to train the base model is derived from TransferZ [ 29 ] with the addition of 5-band images from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) Second Public Data Release [ 1 ] and additional quality cuts as specified in [ 10 ] . Figure 1: The redshift number distributions of the 557 common galaxies using spectroscopic ground truth from GalaxiesML (blue) and photometric ground truth from TranferZ-Images (red). This results in a dataset we call TransferZ-Images 1 1 1 TransferZ-Images is available on Zenodo here. consisting of 100,442 galaxies with 5-band g, r, i, z, y images. We use the GalaxiesML [ 10 ] 2 2 2 GalaxiesML is available on Zenodo here. dataset for fine-tuning. These datasets have been used for redshift estimation in the context of cosmology, and are ML-ready and complement each other by covering different galaxy color types and redshift ranges. The TransferZ-Images dataset has redshift ground truths from photometric redshift estimates while GalaxiesML has ground truth from spectroscopy. The TransferZ-Images photometric redshifts are less accurate than GalaxiesML but contain a fainter and larger variation in galaxy types. Ideally, it is a good dataset to train the base model. We aim to fine-tune on GalaxiesML to improve the base model with more accurate redshifts. In principle, the combination should produce a model that is more accurate and generalizable. We additionally create the Combo dataset by combining TransferZ-Images and GalaxiesML, resulting in a dataset containing 386,286 galaxies. For the 557 common galaxies in both datasets we use the GalaxiesML spectroscopic redshift as ground truth. Additionally, comparing the spectroscopic and photometric ground truth redshifts for the overlapping galaxies, we find no systematic bias in both redshift distributions (see Fig. 1 ). The data used in this study is summarized in Table 1 . The three datasets are split into 80 \% training, 10 \% validation and 10 \% testing subsets. Table 1: Data Summary. Dataset Ground No. of z Median z i-mag No. Truth z Sources 90th \% ile Uncertainty

Read the complete text