Measuring Embedding Sensitivity to Authorial Style in French: Comparing Literary Texts with Language Model Rewritings
Benjamin Icard, Lila Sainero, Alice Breton, Evangelia Zve, Jean-Gabriel Ganascia · 2026
Read Measuring Embedding Sensitivity to Authorial Style in French: Comparing Literary Texts with Language Model Rewritings by Benjamin Icard, Lila Sainero, Alice Breton, Evangelia Zve, Jean-Gabriel Ganascia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10606.txt.
Introduction
Benjamin Icard and four colleagues measure how much author style survives in text embeddings. Large language models can convincingly copy human writing styles, yet it is unclear how much style data is stored in embeddings from any language model and kept after LLM rewriting. Their 2026 paper tests these questions in French, using a controlled literary dataset to measure style shifts via changes in embedding spread.
The authors gather French excerpts from Tufféry, Proust, Céline, and Yourcenar, plus GPT-, Mistral-, and Gemini-based style copies under fixed-topic conditions. Encoding all texts with thirteen embedding models, they measure overall style effects on embedding spread across human authors and copied texts. Embeddings reliably capture author style features, and these signals persist after rewriting, while also showing LLM-specific patterns. The results offer good leads for spotting authorship imitation in the era of language models.
Catalog record
| CREATOR | Benjamin Icard, Lila Sainero, Alice Breton, Evangelia Zve, Jean-Gabriel Ganascia |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10606
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Measuring Embedding Sensitivity to Authorial Style in French: Comparing Literary Texts with Language Model Rewritings Benjamin Icard, Lila Sainero, Alice Breton, Evangelia Zve, Jean-Gabriel Ganascia Abstract Large language models (LLMs) can convincingly imitate human writing styles, yet it remains unclear how much stylistic information is encoded in embeddings from any language model and retained after LLM rewriting. We investigate these questions in French, using a controlled literary dataset to quantify the effect of stylistic variation via changes in embedding dispersion. We observe that embeddings reliably capture authorial stylistic features and that these signals persist after rewriting, while also exhibiting LLM-specific patterns. These analytical results offer promising directions for authorship imitation detection in the era of language models. 1 Introduction Computational stylometry has long been central to digital humanities, using natural language processing (NLP) for authorship attribution and stylistic comparison 50 ; 28 . This has involved quantifying inter-textual distance and developing feature-based methods to support author verification 47 ; 46 ; 7 . In recent years, embedding methods, particularly Transformer-based contextual models ( 54 ; 15 ) , have improved literary authorship attribution through richer textual representations of authorial profiles ( 52 ; 27 ) . Yet embedding-based analysis often centers on semantic tasks, especially topic modeling ( 1 ) and content similarity ( 43 ) , rather than style in embeddings, aside from a few studies ( 57 ; 25 ) . Assessing more systematically how embeddings capture authorial style would improve the explainability of embedding representations and strengthen authorship characterization and attribution in computational stylometry. The increasing capabilities of large language models (LLMs) to imitate human-authored styles make this issue more urgent. Recent work on style transfer shows that LLMs can reproduce salient stylistic features in literary settings ( 36 ) , but also that they still struggle to imitate more implicit writing styles in extra-literary settings ( 56 ) . Because LLMs are language models, this reinforces the need to assess the sensitivity of embedding vectors to style, and the extent to which LLMs preserve or alter that sensitivity during style transfer ( 24 ) . This paper reports a controlled experiment on French literary texts evaluating embedding sensitivity to target stylistic features in human-authored texts and their LLM imitations. We compile a dataset of French excerpts from Tufféry 53 , Proust 39 , Céline 9 , and Yourcenar 58 , together with GPT-, Mistral-, and Gemini-based stylistic imitations under fixed-topic conditions. We encode all texts with thirteen embedding models and quantify aggregated stylistic effects on embedding dispersion across human authors and imitated texts. Section 2 reviews prior work on embedding sensitivity to writing style, with particular attention to the human–LLM distinction. Section 3 introduces our French literary dataset and the evaluation of style transfer under LLM generation. Section 4 presents the vectorization of the dataset with thirteen embedding models, a structural evaluation with clustering, and results on how writing style affect embedding dispersion in human-authored texts and LLM imitations. Section 5 examines LLM-specific effects on embedding sensitivity, bringing explainability to the style transfer evaluation used to validate the dataset. Finally, Section 6 concludes and outlines directions for future work. All reproducibility materials and results are available at: https://github.com/sma-libra/style-embedding-sensitivity 2 Related Work Feature-Based Stylometry. Computational stylometry classically relies on surface features such as word frequencies, character n -grams ( 8 ; 42 ) and punctuation patterns ( 16 ) , often represented with TF-IDF term weighting ( 44 ; 5 ) , to capture authorial voice ( 55 ; 19 ; 33 ) . Style Embeddings. More recent work examines whether embedding-based representations also encode feature-based stylistic information ( 30 ) . 52 propose an author-verification framework that tests whether embedding spaces encode stylistic features rather than mainly semantic content. Using this framework, they show that specialized Doc2Vec-based author embeddings ( 29 ; 17 ; 32 ) are often more semantically driven, while simpler pretrained sentence encoders such as USE-DAN ( 10 ) and SBERT ( 41 ) can perform better on stylistic feature families. Measuring Sensitivity. A related line of work addresses style-semantics conflation more directly through topic-controlled or style-sensitive embedding representations. More specifically, 57 show that content-controlled training improves style-topic separation in BERT-based representations. Adjacent work by 11 shows that style-sensitive encoders can support efficient detection of stylistic shifts in multi-author documents. In parallel, 38 introduce the style embedding model LISA whose dimensions correspond to stylistic features. Most directly related to our approach, 25 use Queneau’s fixed-topic variations to measure embedding sensitivity to style under rewriting by a single LLM, rather than focusing specifically on author-defining features and their preservation. LLM Style Transfer. Recent research on LLMs’ representation and high-fidelity transfer of literary style is motivated by their demonstrated capacity to imitate authorial writing. In literary contexts, 36 show that GPT-4o reproduces salient authorial stylistic features under thematic control, while 45 and 20 show that literary style is reflected in LLM internal representations. In parallel, 23 study LLMs for authorship analysis, while 21 propose TinyStyler, a lightweight approach to few-shot style transfer conditioned on author embeddings. Taken together, these works have examined embedding sensitivity to style in smaller language models and, more recently, in LLMs. However, little work has investigated whether embeddings are reliably sensitive to the defining stylistic features of specific authors, and to what extent this sensitivity remains measurable after LLM rewriting. We investigate this question in French using stylistically marked and diverse literary texts. 3 Dataset 3.1 Text Materials We assembled a dataset of 1,248 French literary texts that combines human-authored originals with LLM-generated imitations that reproduce the human authors’ styles under fixed-topic conditions. Figure 1: Textual corpora and generation scheme used to construct the StyloGen embedding dataset. The reference corpora are Tuffery_ref (single topic, various styles) and Style_ref (various topics, single style per author). Three LLMs generate Style_gen by rewriting Tufféry texts in the styles of the authors in Style_ref , using the French prompt shown. 3.1.1 Reference Corpus We first compiled a reference corpus of 384 French original literary texts, separated in two groups according to topic and style variation. The first group, Tuffery_ref , consists of 96 texts extracted from Stéphane Tufféry’s Le style mode d’emploi 53 . 1 1 1 We excluded 3 of the 99 texts from Tufféry’s Le style mode d’emploi when constructing Tuffery_ref because they departed from the bus-journey topic. We use this collection because, like Raymond Queneau’s Exercices de style ( 40 ) used in 25 , it keeps a single topic across all texts: the story of a bus journey in Paris. But unlike Queneau’s variations, however, Tufféry structures stylistic variation more systematically, including explicit pastiches of major French authors such as Balzac, Flaubert, and Hugo. Tuffery_ref therefore provides a fixed-topic, style-diverse baseline for stylometric comparison. The second group, Style_ref , comprises 288 texts of length comparable to Tuffery_ref , evenly distributed across three subclasses: • Proust_ref : 96 texts from Proust’s Du côté de chez Swann , the first volume of À la recherche du temps perdu 39 ; • Celine_ref : 96 texts from Céline’s Voyage au bout de la nuit 9 ; • Yourcenar_ref : 96 texts from Yourcenar’s Mémoires d’Hadrien 58 . The authors were selected to maximize stylistic