Learning What’s Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics

Pablo Mercader-Perez, Carolina Cuesta-Lazaro, Daniel Muthukrishna, Jeroen Audenaert, V. Ashley Villar, David W. Hogg, Marc Huertas-Company, William T. Freeman · 2026

Read Learning What’s Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics by Pablo Mercader-Perez, Carolina Cuesta-Lazaro, Daniel Muthukrishna, Jeroen Audenaert, V. Ashley Villar, David W. Hogg, Marc Huertas-Company, William T. Freeman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-09787.txt.

Introduction

Mercader-Perez and seven co-authors propose a learning method that splits real signals from sensor flaws across sensors. Seen data mix signals with sensor flaws that confound physics reading and tangle joining views. The method uses shared views, a dual-encoder design, and a what-if goal to split inner signals from warps and noise. These codes support what-if view making, value guessing free of warps, and tool-free matching search. Galaxy images from DESI Legacy and Hyper Suprime-Cam show it in a multi-tool setting.

Catalog record

CREATORPablo Mercader-Perez, Carolina Cuesta-Lazaro, Daniel Muthukrishna, Jeroen Audenaert, V. Ashley Villar, David W. Hogg, Marc Huertas-Company, William T. Freeman
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45320
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.09787

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning What’s Real: Disentangling Signal and Measurement Artifacts in Multi-Sensor Data, with Applications to Astrophysics Pablo Mercader-Perez, Carolina Cuesta-Lazaro, Daniel Muthukrishna, Jeroen Audenaert, V. Ashley Villar, David W. Hogg, Marc Huertas-Company, William T. Freeman Abstract Data collected from the physical world is always a combination of multiple sources: an underlying signal from the physical process of interest and a signal from measurement-dependent artifacts from the sensor or instrument. This secondary signal acts as a confounding factor, limiting our ability to extract information about the physics underlying the phenomena we observe. Furthermore, it complicates the combination of observations in heterogeneous or multi-instrument settings. We propose a deep learning framework that leverages overlapping observations, a dual-encoder architecture, and a counterfactual generation objective to disentangle these factors of variation. The resulting representations explicitly separate intrinsic signals from sensor-specific distortions and noise, and can be used for counterfactual view generation, parameter inference unconfounded by measurement distortions, and instrument-independent similarity search. We demonstrate the effectiveness of our approach on astrophysical galaxy images from the DESI Legacy Imaging Survey (Legacy) and the Hyper Suprime-Cam (HSC) Survey as a representative multi-instrument setting. This framework provides a general recipe for scientific and multi-modal self-supervised pretraining: construct training pairs from overlapping observations of the same physical system, treat sensor- or modality-specific effects as augmentations, and learn invariant representations through counterfactual generation. 1 Introduction Observational data collected from the physical world can be viewed as the result of a causal process. First, an event of interest occurs, generating a physical signal. For example, a distant pulsating star emits electromagnetic waves, a person speaking creates pressure waves in the air, and a beating heart produces pressure waves that propagate through blood vessels. This signal cannot be directly observed. Instead, it must pass through a sensor or instrument that records it. Any instrument used introduces measurement artifacts, often referred to as instrument systematics or bias. These effects include transformations like the nonlinearity of Charge-Coupled Devices (CCD) in a camera or a microphone’s frequency response. These act as confounding factors. However, with sufficient understanding of the measurement system, they can often be partially modeled and disentangled from the underlying signal. Additionally, there is a contribution from noise and uncertainty. These are stochastic contributions that limit measurement precision. We treat this noise as stochastic residuals that are not explained by the latent variables learned from data. In other words, \text{Observation}=f(\text{Signal},\text{Instrument})+\text{Noise} . Sometimes f is simple, such as a straightforward point-spread function (PSF) convolution requiring only a good characterization of the optics. However, it often cannot be fully modeled analytically. The true signal is explained by variables that are independent of the measurement and contain the scientifically relevant information; we refer to these as physics variables . The instrumental effects arise from measurement distortions and act as confounders; we refer to these as instrument variables . We present a general approach to learning to disentangle physics and instrument variables in settings where overlapping observations of the same physical system are available from different instruments. The key insight is that cross-matched observations form natural pairs: the same underlying physics observed under different instrument conditions. This provides a training signal for separating the two factors without requiring an explicit forward model of either instrument. We assume only black-box access to the instruments. We demonstrate our approach using astrophysical galaxy images, a domain where modeling instrument systematics is critical. Photons emitted by a distant source must travel through cosmic dust and the Earth’s atmosphere before being recorded by ground-based telescopes, each with its own unique optical system, detector characteristics, and observing conditions. As a result, the same galaxy observed by two different instruments can appear remarkably different due to differences in resolution, noise, and calibration. Historically, obtaining observations of the same phenomena with multiple instruments has been essential for distinguishing real physical signals from instrumental artifacts. For example, in the case of gravitational waves, the signal is buried deeply in detector noise, and coincident observations by multiple instruments are required to confirm detection. In this work, we train our model on {\sim}100{,}000 cross-matched galaxy images from two different ground-based telescopes and show that we learn disentangled representations separating the intrinsic properties of galaxies from the instrument-specific effects, enabling robust inference of galaxy properties, counterfactual generation across instruments, and instrument-independent similarity search. Figure 1: Counterfactual Reconstruction Framework. The model learns to disentangle intrinsic galaxy properties from instrument systematics by reconstructing an anchor image via a dual-encoder architecture and conditional flow matching. Training uses data triplets consisting of: an anchor observation (signal s from instrument i ), an instrument-augmented observation ( same source s , a different instrument i^{\prime} ), and a physics-augmented observation (a different source s^{\prime} , same instrument i ). The physics encoder extracts latents from the instrument-augmented pair to capture invariant physical features, while the instrument encoder processes the physics-augmented observation to isolate measurement artifacts. These latent variables jointly condition the flow-matching decoder. Crucially, the anchor image is never fed into the encoders; it is used only as a target for the flow-matching loss, forcing the model to generate the reconstruction counterfactually. 2 Related work Disentangled representation learning. Learning representations that separate independent factors of variation is a longstanding goal in representation learning. Early work on variational autoencoders (VAEs), such as \beta -VAE ( 13 ) , encouraged disentanglement by regularizing the latent space via the loss function. A natural source of structure is multi-view (or multi-instrument) observations, where the same underlying content is observed under varying conditions or at different times. 6 proposed separate encoders for time-variant and time-invariant components in video, while 27 used a dual-encoder architecture for pose-invariant face recognition. From a causal perspective, 25 framed disentanglement as the recovery of independent causal mechanisms, connecting representation learning and causal inference. Our framework draws on these ideas but enforces structural disentanglement through a specific architecture-driven information bottleneck and a counterfactual generation objective, rather than relying on hand-engineered loss functions. Foundation models in the Sciences. Pre-training self-supervised representations on large scientific datasets has recently seen success in different areas of science like biology ( 17 ; 22 , e.g.,) and satellite imagery ( 5 , e.g.,) . In astrophysics, 29 demonstrated the value of learned self-supervised representations for downstream tasks. More recently, AION-1 ( 20 ) , developed a large-scale foundation model trained via masked modeling across different modalities such as imaging and spectra from the Multimodal Universe dataset ( 26 ) . However, a limitation common to these approaches is that instrument-specific effects are either ignored or treated as independent modalities without explicit disentanglement. This has practical consequences for downstream tasks. For instance, a model developed for ESA’s Euclid mission ( 9 ) found that instrument systematics, rather than physically interesting sources, dominated the anomalies identified in the latent space. Our work addresses this by building the separation of physics and instrument factors into the architecture itself. Data-driven Noise Models. As an independent and complementary approach to representation learning, several works have tackled modeling instrument effects through explicit data-driven modeling of noise and systematics ( 15 ; 16 , e.g.,) . These require a detailed understanding of the noise structure and perfect separability from the physics. In contrast, our framework treats the instrument as

Read the complete text