Stable but Wrong: An Inference Limit in Galactic Archaeology
Zhipeng Zhang · 2026
Read Stable but Wrong: An Inference Limit in Galactic Archaeology by Zhipeng Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27368.txt.
Introduction
Zhang challenges a core belief of big-data science, that larger samples with smaller errors must converge on the truth, using stellar ages in Galactic archaeology. From a large sample of subgiant stars, he finds a pocket of the quality space set by signal-to-noise ratio and parallax precision. There, inferred formation timescales sit systematically off an independent asteroseismic reference. Because the statistical errors there stay small, the result looks stable while remaining wrong by roughly half a billion to a billion years.
Catalog record
| CREATOR | Zhipeng Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56773 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.27368
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Stable but Wrong: An Inference Limit in Galactic Archaeology Zhipeng Zhang Abstract Statistical inference in observational science typically relies on a fundamental assumption: as sample size increases and uncertainties decrease, the inferred results should converge to the true physical quantities. This assumption underpins the notion that “big data lead to more reliable conclusions”. In Galactic archaeology, stellar ages inferred from spectroscopic surveys are widely used to reconstruct the formation history of the Milky Way disk. The age–metallicity relation (AMR) and its derived formation timescale are often regarded as key physical diagnostics of early disk evolution. This interpretation carries an implicit premise: that observational quality does not introduce systematic bias into age inference. Here we show that this premise may fail. Using a large sample of subgiant stars, we identify a region in the observational quality parameter space (signal-to-noise ratio and parallax precision) where the inferred formation timescale exhibits a systematic offset of ∼ 0.5 \sim 0.5 – 1 Gyr 1\,\mathrm{Gyr} relative to an independent asteroseismic reference, while the statistical uncertainties remain small, thus producing a “stable-but-wrong” inference state. Through matching-sample analysis and external ground-truth anchoring, we further demonstrate that this bias cannot be explained by sample selection effects, observational noise, or model calibration errors. Rather, it arises from a structural coupling between observational quality and the age inference process. Under identical physical conditions, differing only in observational quality, the inferred age distributions yield systematically different formation histories. The magnitude of this offset is comparable to the timescales that distinguish different scenarios of Galactic disk formation, indicating that this inference bias is large enough to alter the physical interpretation of the Milky Way disk’s formation history. These results reveal a fundamental inference limit: when the key information needed to distinguish true signal from systematic bias is neither contained in the observed data nor captured by the uncertainties, statistical stability does not guarantee physical correctness. Under such conditions, increasing the data volume may instead reinforce systematic errors. This limit may be prevalent in scientific problems that rely on indirect observations and inversion processes. Corresponding author: Zhipeng Zhang ( [email protected] ) Introduction Modern observational science is built upon a core assumption: as sample size increases and statistical uncertainties decrease, inferred results should progressively approach the true physical quantities. This assumption underpins the widespread application of large-scale survey observations and data-driven methods, forming the basis of the notion that “more data yield more reliable conclusions”. However, this premise carries a critical hidden condition: that the observational data contain sufficient discriminative information to distinguish true signals from systematic biases. Once such information is missing or not properly modeled, statistical convergence may merely reflect internal consistency, no longer guaranteeing an accurate characterization of the underlying physical process. Galactic archaeology provides an important testing ground for this issue. The stellar age–metallicity relation (AMR) is widely used to reconstruct the formation history, chemical enrichment timescale, and early formation epoch of the Milky Way disk. Existing studies typically interpret statistically stable AMR structures as direct imprints of physical processes, and accordingly infer the disk formation mode and its evolutionary path. However, stellar ages are not directly observed but rather inferred indirectly from spectroscopy, photometry, parallaxes, and stellar modeling or data-driven pipelines [ 6 , 3 ] . Observational conditions can therefore influence the final results through the inference process, potentially introducing structural biases at the population level [ 5 , 2 ] . Against this background, a key question arises: when inferred results are highly stable in a statistical sense and carry very small uncertainties, can they be regarded as reliable reconstructions of the true formation history? In this paper, we show that the answer is negative. We identify a region in the observational parameter space (signal-to-noise ratio and parallax precision) where inferred ages exhibit remarkable statistical stability yet show systematic offsets relative to [ 4 ] . This phenomenon manifests as stable-but-wrong : the results are internally statistically self-consistent but deviate from external physical ground truth. To identify and validate this behavior, we construct a diagnostic framework based on observational quality and introduce a dimensionless metric to quantify the significance of the bias relative to statistical uncertainties. Simultaneously, we anchor external ground truth using APOKASC asteroseismic ages and control for observable physical covariates via coarsened exact matching (CEM), thereby excluding the effects of sample selection and population differences. On this basis, we compare the age structures and derived formation histories obtained from inferred ages versus asteroseismic ages under different observational conditions. Our results show that the stable-but-wrong phenomenon cannot be explained by sample selection, observational noise, uncertainty underestimation, or simple model calibration errors. Instead, it arises from a structural coupling between observational quality and the age inference process. Under identical physical conditions, merely changing the observational quality leads to systematic changes in the inferred age distribution and the resulting formation history. This bias manifests not only at the level of individual stars but also in population-level statistics, including key physical diagnostics such as formation timescale, AMR slope, and formation time at fixed metallicity. More importantly, the magnitude of this bias is comparable to the timescales that distinguish different scenarios of Galactic disk formation [ 7 , 1 ] , and is therefore sufficient to alter the physical interpretation of the early formation history of the Milky Way disk. This demonstrates that a formation history derived from the statistical structure of inferred ages, while statistically highly stable, may systematically deviate from the true evolutionary path. The above results elevate the issue from an empirical bias to a fundamental limitation at the inference level: when the key information needed to distinguish true signals from systematic biases is not contained in the observational data and is not explicitly captured by the uncertainty model, the inference process can stably converge to an incorrect solution. Under such conditions, increasing the data volume or reducing statistical errors does not automatically improve physical accuracy; it may instead reinforce the stability of erroneous conclusions. Thus, stable-but-wrong is not merely an empirical phenomenon in stellar age inference, but rather an inference limit determined by the information structure. This limit may be prevalent in scientific problems that rely on indirect observations and inversion processes, and it suggests that, when interpreting large-scale observational data, the relationship between statistical stability and physical correctness needs to be critically re-examined. 1 Data and Methods 1.1 Overall Framework: From Observational Bias to Inference Limit The central goal of this paper is not merely to describe the age–metallicity structure, but to test a more fundamental question: when observational conditions couple with the inference process, can statistical convergence still guarantee physical correctness? To this end, we construct a diagnostic framework centered on the “bias–uncertainty contrast”, and gradually rule out common explanations using external ground truth and matched control experiments, thereby elevating empirical bias to a structural limitation at the inference level. 1.2 Observational Data and Sample Definition The analysis sample is drawn from large-scale spectroscopic surveys, combined with parallax and photometric information to construct the stellar parameter space. To ensure statistical robustness, we first apply basic quality cuts, including signal-to-noise ratio (SNR), effective temperature and surface gravity ranges, and parallax measurement accuracy ( \varpi/\sigma_{\varpi} ). These cuts are used to construct a unified baseline