TAB-AUDIT: Detecting AI-Fabricated Scientific Tables via Multi-View Likelihood Mismatch

Shuo Huang, Yan Peng, Lizhen Qu · 2026

Read TAB-AUDIT: Detecting AI-Fabricated Scientific Tables via Multi-View Likelihood Mismatch by Shuo Huang, Yan Peng, Lizhen Qu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19712.txt.

Introduction

Huang, Pen, and Qu present the first systematic study of fake scientific tables made by AI. They build FabTab, a benchmark of 1,173 fabricated papers and 1,215 human-written ones. All come from empirical work in language technology. Their TAB-AUDIT framework scores tables with several feature views. One key signal is a mismatch between a table's frame and its numbers. A random forest built on these features reached 0.987 AUROC in-domain. It also scored 0.883 on out-of-domain tests.

Catalog record

CREATORShuo Huang, Yan Peng, Lizhen Qu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59456
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19712

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TAB-AUDIT: Detecting AI-Fabricated Scientific Tables via Multi-View Likelihood Mismatch Shuo Huang, Yan Peng, Lizhen Qu Abstract AI-generated fabricated scientific manuscripts raise growing concerns with large-scale breaches of academic integrity. In this work, we present the first systematic study on detecting AI-generated fabricated scientific tables in empirical NLP papers, as information in tables serve as critical evidence for claims. We construct FabTab , the first benchmark dataset of fabricated manuscripts with tables, comprising 1,173 AI-generated papers and 1,215 human-authored ones in empirical NLP. Through a comprehensive analysis, we identify systematic differences between fabricated and real tables and operationalize them into a set of discriminative features within the TAB-AUDIT framework. The key feature, within-table mismatch , captures the perplexity gap between a table’s skeleton and its numerical content. Experimental results show that RandomForest built on these features significantly outperform prior state-of-the-art methods, achieving 0.987 AUROC in-domain and 0.883 AUROC out-of-domain. Our findings highlight experimental tables as a critical forensic signal for detecting AI-generated scientific fraud and provide a new benchmark for future research 3 3 3 The source code and dataset are available at https://anonymous.4open.science/r/AI_paper_detection-3DCB/ . 1 Introduction Large language models (LLMs) have introduced new avenues for breaching academic integrity by enabling the rapid generation of convincing but fabricated scientific manuscripts. Qualitative studies published in Nature ( 9 ) and ( 13 ) have shown that all major LLMs can potentially assist in producing fraudulent papers at scale , and the resulting outputs can be sufficiently realistic to evade detection, raising serious concerns about the integrity and trustworthiness of the scientific literature ( 10 ; 5 ; 17 ) . Detecting AI-generated fraudulent manuscripts is increasingly challenging, as LLMs are widely used to assist with writing, clarity, and various stages of the research process ( 19 ) . Recent advances in automated “AI scientists” ( 21 ; 6 ) further blur the boundary by enabling end-to-end automation of the research pipeline, from literature review to experimentation and manuscript drafting. Consequently, distinguishing fabricated AI-generated content from legitimate AI-assisted or AI-produced research becomes difficult. Inspired by prior work on detecting ghost citations ( 20 ) , we focus on detecting AI-generated fabricated tables as a key signal of fabricated papers, since experimental tables provide transparent empirical evidence for claims and enable comparison across methods; fabricating such information fundamentally undermines the credibility of the entire manuscript. Despite growing concerns about AI-generated fraudulent content in scientific research, there has been no systematic study on evaluating and detecting AI-generated fabricated experimental tables. Prior work has primarily focused on textual artifacts, such as hallucinated citations or generated narratives ( 8 ; 10 ) , while largely overlooking structured empirical evidence in tables. Furthermore, the lack of quantitative analyses and benchmark datasets limits our ability to characterize the prevalence, patterns, and detectability of fabricated tables, thereby hindering the development of reliable detection methods. Model P R F1 BERT-base 72.4 70.8 71.6 SpanBERT 76.9 75.8 76.3 Ours 98.7 98.4 98.6 Blue = skeleton (headers/row labels) Red = numeric tokens Mismatch \uparrow : typical skeleton , atypical 98.x digits. Figure 1: Compact illustration of the table skeleton–numeric mismatch. To enable systematic investigation and evaluation of AI-fabricated tables, we construct the first benchmark dataset of fab ricated manuscripts with tab les, named FabTab , comprising 1,173 AI-generated papers and 1,215 human-authored ones in empirical NLP. We focus on this paper type because tables are the primary medium for presenting evidence supporting key claims. We then conduct a comprehensive analysis of the differences between fabricated and real tables and operationalize those findings into a set of highly discriminative features in the framework TAB-AUDIT . We explore the inherent inconsistency between table skeleton and numeric digits of the table as shown in Figure. 1 . Based on these features, we show that traditional ML models significantly outperform prior SOTA methods ( 11 ; 2 ) by a wide margin. Our key contributions are three-fold: • We construct FabTab using a dedicated pipeline that leverages literature on the same topics. The in-domain set comprises 1,012 recent human-authored papers and 976 fabricated ones generated by GPT-4o conditioned on existing literature. To reflect real-world difficulty, each generated paper is paired with real papers on closely related topics. In total, the in-domain set contains 5,284 tables. To evaluate generalization to unseen generation models, we build an out-of-domain test set using GPT-5.2 , consisting of 197 fabricated papers and 203 human-authored ones on highly similar topics. A human study (§ 2.4 ) further shows that papers generated by our pipeline are difficult to distinguish from real ones. • Based on a comprehensive analysis of the dataset, we operationalize our findings into a set of highly discriminative features within the TAB-AUDIT framework. The most salient feature is within-table mismatch , which captures the perplexity gap between a table’s structural skeleton and its numerical content. We also incorporate additional numerical features that do not rely on language models, which further enhance detection performance. • We train traditional machine learning models on our features aggregated at the paper level and evaluate them on both in-domain and out-of-domain test sets. Among all models, RandomForest ( 18 ) achieves the best performance, with 0.987 AUROC on the in-domain test set and 0.883 AUROC on the out-of-domain test set. We further show that our models achieve higher AUROC when detecting fabricated tables generated by simple prompting of LLMs compared to those produced by our paper generation pipeline. 2 Benchmark Construction and Paper Generation Our benchmark is designed to isolate provenance rather than topic. The human papers are composed of recent experimental NLP papers, while the fabricated arm is generated to match the same broad domain and paper format. Without it, a detector could separate classes using stale venue conventions or cross-domain artifacts instead of signals tied to fabricated empirical evidence. 2.1 Collecting Recent Experimental NLP Papers We draw the human corpus from arXiv rather than from a heterogeneous mixture of venues or historical snapshots. To reduce the chance that benchmark papers were already absorbed into the pretraining data of the language models used in generation or auditing, we restrict the source pool to papers posted within the month immediately preceding collection. This recency constraint also keeps the benchmark close to a single publication regime, limiting drift in formatting conventions, writing style, and topical emphasis. We further restrict the source pool to experimental NLP papers . This avoids mixing result tables from substantially different scientific genres, such as purely theoretical work, surveys, or papers from unrelated domains whose tables obey different reporting conventions. In the indexed human source pool, this procedure yields 1,118 candidate papers. After table extraction and validity filtering, 1,012 human papers remain in the clean benchmark used. This collection strategy matters for two reasons. It reduces domain bias: a detector should not win simply because fabricated papers come from a different field or historical period. Second, it raises the scientific standard of the benchmark: if a detector succeeds here, it is more likely to be detecting signals tied to fabricated empirical evidence rather than superficial topic or formatting artifacts. 2.2 Literature-Grounded Fabricated Paper Generation To construct realistic yet controllable fabricated papers—primarily to obtain realistic fabricated tables —we design a literature-grounded pipeline for empirical

Read the complete text