Sanity Checks for Agentic Data Science

Zachary T. Rewolinski, Austin V. Zane, Hao Huang, Chandan Singh, Chenglong Wang, Jianfeng Gao, Bin Yu · 2026

Read Sanity Checks for Agentic Data Science by Zachary T. Rewolinski, Austin V. Zane, Hao Huang, Chandan Singh, Chenglong Wang, Jianfeng Gao, Bin Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11003.txt.

Introduction

Rewolinski and six colleagues propose two lightweight sanity checks for agentic data science, grounded in the PCS framework. Agentic systems can reach overly positive conclusions that users struggle to detect. The checks use changes to test whether an agent tells signal from noise. They can expose unsupported positive claims. Together they show whether an output found stable signal, responded to noise, or is sensitive to chance details. On eleven real datasets with OpenAI Codex, a positive conclusion proves unsupported in six, while reported confidence proves poorly matched to steadiness.

Catalog record

CREATORZachary T. Rewolinski, Austin V. Zane, Hao Huang, Chandan Singh, Chenglong Wang, Jianfeng Gao, Bin Yu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63454
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11003

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sanity Checks for Agentic Data Science Zachary T. Rewolinski, Austin V. Zane, Hao Huang, Chandan Singh, Chenglong Wang, Jianfeng Gao, Bin Yu Abstract Agentic data science (ADS) pipelines have grown rapidly in both capability and adoption, with systems such as OpenAI Codex now able to directly analyze datasets and produce answers to statistical questions. However, these systems can reach falsely optimistic conclusions that are difficult for users to detect. To address this, we propose a pair of lightweight sanity checks grounded in the Predictability-Computability-Stability (PCS) framework for veridical data science. These checks use reasonable perturbations to screen whether an agent can reliably distinguish signal from noise, acting as a falsifiability constraint that can expose affirmative conclusions as unsupported. Together, the two checks characterize the trustworthiness of an ADS output, e.g. whether it has found stable signal, is responding to noise, or is sensitive to incidental aspects of the input. We validate the approach on synthetic data with controlled signal-to-noise ratios, confirming that the sanity checks track ground-truth signal strength. We then demonstrate the checks on 11 real-world datasets using OpenAI Codex, characterizing the trustworthiness of each conclusion and finding that in 6 of the datasets an affirmative conclusion is not well-supported, even though a single ADS run may support one. We further analyze failure modes of ADS systems and find that ADS self-reported confidence is poorly calibrated to the empirical stability of its conclusions 1 1 1 All relevant code can be found at https://github.com/zachrewolinski/stat-genie . 1 Introduction Agentic data-science (ADS) pipelines are becoming increasingly powerful ( 23 ; 40 ) , driven in part by the rapid spread of autonomous systems such as OpenAI Codex and Claude Code. These pipelines are capable of taking as input a natural-language query and a relevant dataset and executing the full analytical workflow to produce an answer, enabling end-to-end “vibe data science.” Any user can obtain a polished analysis, complete with model selection, hypothesis tests, and a written summary. However, the conclusions produced by ADS pipelines can be wrong in ways that are difficult to detect. An agent may make an unreasonable data-cleaning decision, choose an inappropriate model, or overlook confounders, but the resulting report will still appear polished and defensible ( 8 ) . Worse, for difficult datasets, a handful of runs can yield an entirely different conclusion than a full stability analysis, underscoring the need for systematic checks ( Fig. 3 ). Thus, the rapid adoption of ADS pipelines poses an urgent risk to flooding science with unreliable findings ( 63 ) . Existing solutions to tackle this problem have limitations. Classical tests of statistical significance generally fail in agentic settings. For example, we observe that agents handle statistical evidence poorly in practice, over-interpreting estimates that lack significance and “ p -hacking” by anchoring on whichever test is most favorable ( Section 4.3 ). Alternatively, simply asking an agent to assess the uncertainty of an analysis also does not succeed, as agent-reported confidence is poorly calibrated to the actual stability of an underlying result ( Section 5.3 ). Finally, “multiverse” analyses enable quantifying the impact of different choices in an ADS pipeline, but their outputs quickly become too large for human review, creating a need for actionable checks ( 8 ; 46 ) . To tackle this challenge, we make use of the Predictability-Computability-Stability (PCS) framework for veridical data science ( 74 ; 73 ; 45 ) . It establishes that trustworthy, reality-checked conclusions must be stable under reasonable perturbations across different data science stages, such as problem specification, data cleaning, and model choices. In this spirit, we generate small, reasonable changes, which we call PCS perturbations . These perturbations probe agentic failure modes which would not impact a competent human analyst’s conclusions. If an agent’s conclusions change dramatically under these perturbations, they are not trustworthy. Consistency under these perturbations is necessary but not sufficient, since an agent could also arrive at a stable but biased conclusion. Since ground-truth conclusions are rarely known in data science, we use a null-defining perturbation to put the agent in a setting where the answer is known and testable. For example, we may remove all the signal in the data (e.g., independently shuffling each column) to establish a negative control against which we can measure the agent’s capabilities. We then run the agent many times to generate two empirical distributions: one where both null-defining and PCS perturbations are applied, which we call the null distribution , and another where only PCS perturbations are applied, which we call the alternative distribution . Using these two distributions, we apply two complementary sanity checks (see Fig. 1 ): (1) The Yes check : a one-sample bootstrap test of whether the ADS mean response on the original data is significantly above the midpoint of the scale (i.e., in the “Yes” range), thereby testing whether a positive conclusion was reached by chance. (2) The Overlap check : quantifies the similarity between the null and alternative distributions, thereby testing for bias in the ADS conclusion that does not come from signal in the data. In experiments using OpenAI Codex, we first validate the approach on simulations with controlled signal-to-noise ratios, confirming that the checks track ground-truth signal strength ( Section 4.2 ). We then apply them to eleven real-world datasets from the BLADE benchmark, where the checks characterize each dataset’s signal regime and reveal one case in which the agent’s affirmative conclusion is not well-separated from its behavior on pure noise ( Section 4.3 ). We further analyze failure modes of ADS systems ( Section 5 ), showing that ADS self-reported confidence is poorly calibrated to the empirical stability of its conclusions and characterizing how different steps fail using the BLADE agentic harness (an alternative to Codex). We view this work as a first step toward evaluation standards for agentic data science, grounded in the core principles of the PCS framework for veridical data science. Figure 1: Pipeline for computing the PCS sanity checks. The user supplies a dataset and question. The pipeline then generates perturbed variants of these under both null-defining and PCS perturbations. The coding agent analyzes each independently, producing null and alternative response distributions. A bootstrap test & overlap coefficient then inform whether the agent found a data-grounded, stable positive conclusion. 2 Related Work Agentic data science (ADS). The risks of AI automation in the scientific workflows have been documented in fields adjacent to data science, including LLM-hacking for annotation ( 7 ; 66 ) and broader stages of the research pipeline ( 34 ) . More directly relevant, 3 demonstrates that guardrails are necessary to prevent humans from p -hacking using LLM-driven data science, while other work highlights the value of transparency in intermediate data-science artifacts as a partial remedy ( 58 ; 40 ) . Despite these warning signs, much of the ADS literature has focused on capability and benchmarking, including evaluation suites such as DSGym ( 41 ) , ScienceAgentBench ( 10 ) , IDA-Bench ( 28 ) and others ( 59 ; 30 ) , alongside accompanying agentic frameworks ( 23 ) . Comparatively, little attention has been paid to whether ADS conclusions can be trusted, motivating the need for intuitive and actionable sanity

Read the complete text