Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction

Alexander Smirnov · 2026

Read Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction by Alexander Smirnov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21653.txt.

Introduction

Alexander Smirnov argues fine-tuned AI-text detectors amplify a pretrained direction, not an AI-human boundary. Detectors flag formal writing as machine-generated, rating the essay as ninety-nine point five percent AI but clearing hot AI at ten point five percent. Decomposing a detector into an inherited reading and a tuned residual, the inherited part carries cross-generator transfer and drives the flagging, while the residual stays generator-specific. This yields a no-go: no training objective, threshold, erasure, or ensemble tested removes the harm while preserving detection.

Catalog record

CREATORAlexander Smirnov
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54751
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.21653

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction Alexander Smirnov Abstract AI text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary. On raw encoders before any task supervision, projecting onto centroid(AI) − - centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806 / 0.944 / 0.834 0.806/0.944/0.834 across three architectures ( 86 86 – 106 % 106\% of the fine-tuned discrimination ceiling: on RoBERTa-base, raw projection exceeds fine-tuning); on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested. The same axis inverts on non-native ESL writing (AUROC 0.06 0.06 – 0.20 0.20 ) a falsifiable prediction unique to the typicality reading. A 24 24 -example frozen probe matches full fine-tuning ( 0.900 0.900 vs 0.895 0.895 ). A closed-form Jacobian predictor parameterises axis-manipulating interventions with R 2 = 1.000 R^{2}=1.000 universal, lifts ELECTRA-CE deployment TPR from 0.000 0.000 to 0.904 0.904 at FPR = 1 % \mathrm{FPR}=1\% , and transfers to three independently-trained third-party RoBERTa detectors at 16 / 16 16/16 oracle-equivalence ( 57 % 57\% NYT-FPR reduction on the OpenAI detector). Scope: encoder family; mechanism magnitude HC3-anchored; population-level shared axis with per-text mechanisms varying across architectures. Three operationally distinct probes text-surface caps_rate residualisation, geometric signed- ε \varepsilon ablation, closed-form text-pair predictor agree at cos ⁡ 0.74 / 0.81 / 1.00 \cos 0.74/0.81/1.00 across three architectures, confirming observer-invariance. Under matched-TPR- 0.90 0.90 evaluation, the published intervention zoo (CC, dealign-f2c) is calibration-equivalent across 27 27 cells ( | Δ ​ AUROC | ≤ 0.0081 |\Delta\mathrm{AUROC}|\leq 0.0081 ), and ≥ 97 % \geq 97\% of the LoRA → \to full-FT bias gap on ELECTRA is calibration shift, not learned representation the central claim’s prediction confirmed. 1 Introduction Under canonical ELECTRA cross-entropy fine-tuning, 33.5\% of NYT formal human articles cross the AI decision threshold (threshold \tau=0.5 ); the top decile is scored at P(\mathrm{AI})\geq 0.992 more confidently AI than the average output from any of seven LLM families. Per-family means under the same detector are LLaMA-2-7B 0.71 , Qwen 0.71 , Gemini 0.85 , GPT-4 0.85 , Claude 0.86 , GPT-4o 0.87 , and LLaMA-3 0.90 ; the formal human articles sit above all seven ( 7 ) . This is the gap between what AI-text detectors are designed to do and what they actually rank: under the canonical training recipe, fluent formal humans are flagged at higher confidence than the median output of every modern instruction-tuned model. We argue that the 33.5\% figure is not a calibration bug to be patched but a direct prediction of what fine-tuning a binary classifier on top of a pretrained encoder does to the geometry the encoder already has: fine-tuned AI text detectors do not construct an AI-vs-human boundary, they amplify a pretrained typicality axis. On that reading, fluent formal humans score above the median AI not because the detector misclassified them but because they sit further along the pretrained typicality direction than the HC3 AI population the detector was trained against, and every standard fine-tuning recipe (canonical cross-entropy, CC-loss, dealign-f2c, LoRA, full fine-tuning) calibrates the same fixed axis. The remainder of the paper establishes this in three pillars (§§2, 3, 5) plus a methodological contribution (§4): only an axis-manipulating intervention can close the bias gap, because no thresholding choice on the shared axis can. We organise this paper around three pillars of evidence for that claim and a single methodological contribution: §2 establishes the pretrained typicality axis directly; §3 shows a 24 -example frozen-encoder probe matches full fine-tuning; §4 develops the closed-form per-text predictor that is the paper’s methodological contribution and transfers to three deployed third-party RoBERTa detectors at 16/16 oracle-equivalence (reducing the OpenAI detector’s NYT-FPR by 57\% ); §5 shows three operationally distinct probes measure one axis and that variant-engineering produces calibration shifts rather than learned representations. Beyond the three pillars, §6 demonstrates the recipe extends beyond AI detection to seven other classifier-fairness applications (HANS lexical-overlap NLI, AAVE dialect bias, Bios occupational-gender, and four CC-loss generalisations covering fake-news, toxicity, HateCheck, and MNLI). §7 returns to the thesis after the evidence is in. We turn first to that pretrained typicality axis directly. 2 The Pretrained Typicality Axis Notation. \mathbf{d}_{\mathrm{class}}=\mathrm{centroid}(\mathrm{AI})-\mathrm{centroid}(\mathrm{HC3}) is the raw mean-difference direction in pretrained [CLS] space. Related typicality directions: \mathbf{d}_{\mathrm{typ\_HC3}} (HC3-conditioned, § 4 ), \mathbf{d}_{\mathrm{typ\_A}} (the same axis under the v1 ablation protocol of § 5 ), \mathbf{d}_{\mathrm{typ\_FT}} (post-FT CLS-space variant, § 5 ), \mathbf{d}_{\mathrm{form}} (text-pair contrastive formality direction, § 5 ), and \mathbf{d}_{\mathrm{probe}} (FT decision-head reference direction, § 5 ). Additional variants ( \mathbf{d}_{\mathrm{typ\_NYT}} , \mathbf{d}_{\mathrm{caps\_PLS}} ) are introduced at first use. All directions are unit-normalized. On raw pretrained ELECTRA, the geometric direction \mathbf{d}_{\mathrm{class}}=\mathrm{centroid}(\mathrm{AI})-\mathrm{centroid}(\mathrm{HC3}) 1 1 1 HC3, the Human-ChatGPT Comparison corpus of 5 , is the early-2023 ChatGPT-vs-human reference distribution used as anchor AI population throughout this paper. is what we previously characterise as “typicality-direction, not AI-content-direction” — an axis that misclassifies moderate-perplexity AI text as human within a single in-distribution pool. The signature is sharp: on EvoBench-AI versus EvoBench-human, projecting raw [CLS] at the peak layer onto \mathbf{d}_{\mathrm{class}} gives AUROC 0.302/0.330/0.342 on ELECTRA, RoBERTa-base and DeBERTa-v3 respectively (balanced n=500/500 at random state 42 ). The ELECTRA anchor of 0.292 from the original single-architecture lock reproduces to within \Delta=+0.010 ; the pre-registered falsifier passes. 2 2 2 Ghostbuster within-pool reproduces direction-consistent at \sim 0.50 across all three architectures, consistent with the original ELECTRA-only anchor of 0.467 ; we cite EvoBench inversion as the primary cross-architecture lock and note this paper extends the original ELECTRA-only typicality characterization to three architectures. A direction that scores moderate-perplexity AI less AI than its human counterpart within a single domain cannot be an AI-content axis; it is an axis along which low-perplexity text reads as “typical” regardless of source. The original characterisation we cite was on ELECTRA alone; the three-architecture demonstration in this section is new. The three architectures we analyse in depth (ELECTRA, RoBERTa-base, DeBERTa-v3) sit within a broader cross-architecture support set: \cos(\mathbf{d}_{\mathrm{class}},\mathbf{d}_{\mathrm{form}}) mean 0.71 across 12 encoder backbones (range [0.38,0.98] ); per-text LLaMA-3-8B NLL coupling negative on 15/15 backbones (range -0.244 to -0.820 , all p<10^{-14} ); the §4 closed-form predictor extends to an 8 -encoder panel at R^{2}\geq 0.97 ( |\varepsilon|\leq 0.3 ) and passes 9/9 external-architecture specificity on three held-out backbones \times three seeds. Table 1 is a deep dive within that broader family, not the support set itself. Projecting raw pretrained [CLS] onto \mathbf{d}_{\mathrm{class}} discriminates NYT formal humans from HC3 informal humans at AUROC 0.806/0.944/0.834 across three encoder architectures, attaining 86 – 106\% of the fine-tuned ceiling without any task supervision — and on RoBERTa-base, full fine-tuning reduces discrimination below raw projection on both fluent-formal populations we tested. Across four human populations versus HC3, the raw projection lands 9 positive cells ( \geq 0.70 ), 3 sign-inverted ( \leq 0.21 ), and 0 null out of 12 (Table 1 ) Three-seed fine-tuned pool-AUROC is 0.889\pm 0.002/0.894\pm 0.006/0.967\pm 0.001 on NYT-vs-HC3 across the three architectures, giving \Delta\mathrm{AUROC}_{\mathrm{FT-raw}}=+0.083/-0.050/+0.133 : on RoBERTa-base the raw projection exceeds fine-tuning by 0.050\pm 0.006 at NYT (Table 2 ) . Replication on Ghostbuster RoBERTa-base confirms FT-below-raw at \Delta=-0.018\pm 0.008 . The standardized-effect amplification K_{\mathrm{std}}=(\Delta\mathrm{logit}_{\mathrm{FT}}/\sigma_{\mathrm{FT}})/(\Delta\mathrm{proj}_{\mathrm{raw}}/\sigma_{\mathrm{proj,raw}}) is 1.11/0.86/1.72 three-seed: fine-tuning rescales pretrained geometry by less than 2\times , not constructs new representation. FCE

Read the complete text