Generation, Evaluation, and Explanation of Novelists’ Styles with Single-Token Prompts

Mosab Rezaei, Mina Rajaei Moghadam, Abdul Rahman Shaikh, Hamed Alhoori, Reva Freedman · 2025

Read Generation, Evaluation, and Explanation of Novelists’ Styles with Single-Token Prompts by Mosab Rezaei, Mina Rajaei Moghadam, Abdul Rahman Shaikh, Hamed Alhoori, Reva Freedman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-20459.txt.

Introduction

Rezaei and colleagues teach language models to imitate classic novelists, then grade the fakes. Their 2025 paper works on stylometry, the measurement of writing style. Five nineteenth-century authors anchor the study. They are Dickens, Austen, Twain, Alcott, and Melville. Models were fine-tuned using only single-token prompts as style signals.

Two training routes were compared head to head. Full fine-tuning beat the lighter LoRA method on fidelity. LoRA still offered big efficiency gains in trainable parameters. A DeBERTa classifier graded generated sentences as an automatic judge. At high confidence it agreed with the intended author eighty-two percent of the time. Twain and Melville proved the hardest voices to capture. Syntax comparisons checked parse shapes and sentence lengths. Alcott fakes grew longer parse paths than the real thing. Attention and gradient analyses revealed which cues drive imitation. Human judges confirmed the thematic and linguistic match.

Catalog record

CREATORMosab Rezaei, Mina Rajaei Moghadam, Abdul Rahman Shaikh, Hamed Alhoori, Reva Freedman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-20459

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Generation, Evaluation, and Explanation of Novelists’ Styles with Single-Token Prompts Thanks: This work builds on and substantially extends preliminary research presented at a student research workshop [ 32 ] . Mosab Rezaei, Mina Rajaei Moghadam, Abdul Rahman Shaikh, Hamed Alhoori, Reva Freedman Abstract Recent advances in large language models have created new opportunities for stylometry, the study of writing styles and authorship. Two challenges, however, remain central: training generative models when no paired data exist, and evaluating stylistic text without relying only on human judgment. In this work, we present a framework for both generating and evaluating sentences in the style of 19th-century novelists. Large language models are fine-tuned with minimal, single-token prompts to produce text in the voices of authors such as Dickens, Austen, Twain, Alcott, and Melville. To assess these generative models, we employ a transformer-based detector trained on authentic sentences, using it both as a classifier and as a tool for stylistic explanation. We complement this with syntactic comparisons and explainable AI methods, including attention-based and gradient-based analyses, to identify the linguistic cues that drive stylistic imitation. Our findings show that the generated text reflects the authors’ distinctive patterns and that AI-based evaluation offers a reliable alternative to human assessment. All artifacts of this work are published online. I Introduction The ability to recognize and reproduce an author’s writing style has long fascinated both literary scholars and computer scientists. Stylometry, the quantitative study of writing style, rests on the idea that every author leaves behind unconscious patterns in vocabulary, syntax, and rhythm [ 14 , 42 ] . These patterns have been analyzed for centuries in questions of disputed authorship, the study of literary traditions, and more recently in applications such as security and forensics [ 41 ] . As digital text has grown in scale and diversity, stylometric methods have become increasingly valuable, offering insights not only into literature but also law and education. The expansion of online archives has broadened stylometric research, making it possible to revisit old questions with new computational tools. Digital libraries have played a central role in this progress. Large-scale digitization has made it possible to study thousands of historical works, while also raising questions about how to curate, analyze, and present such collections. Computational tools that detect or generate stylistic variations create opportunities for exploration, pedagogy, and access. For example, a system that generates text in the style of Dickens or Austen could enrich exhibits, classroom teaching, or reading companions that make historical literature engaging for modern audiences. More broadly, this technology shows how digital libraries are evolving from static repositories into dynamic platforms for cultural discovery and scholarly experimentation. These connections show that advances in language technologies are not only technical but also cultural, shaping how heritage is preserved and experienced. Recent advances in large language models shift the focus from style recognition to style generation. The core question is no longer only, Can we tell whether a passage was written by Dickens or Austen? but also, Can a model convincingly write like them? Style-based generation has applications in authorship verification, creative writing, adaptive education, and forensic analysis. Yet two problems remain. First, most data are unpaired, meaning there is no parallel corpus expressing the same content in different authors’ voices. Second, evaluating style is difficult. Metrics such as accuracy or BLEU capture surface resemblance but not deeper stylistic cues, while human judgments are costly and inconsistent. This paper proposes a framework to address these challenges. We fine-tune large language models to generate sentences in the styles of five 19th-century novelists, using single-token prompts as minimal stylistic signals. To evaluate the generated text, we train a transformer-based detector on authentic sentences, using it as both classifier and interpretive tool. We further apply explainable AI (XAI) techniques, including attention- and gradient-based analyses, to uncover the cues that drive stylistic imitation. Finally, we compare two fine-tuning approaches, full fine-tuning and LoRA, to examine trade-offs between efficiency and stylistic fidelity. The dataset 1 1 1 https://huggingface.co/datasets/Mosab-Rezaei/19th-century-novelists and all artifacts 2 2 2 https://github.com/mosabrezaei/Text-Generation-XAI of this work are also publicly available online. This study is guided by two main questions: RQ1: Given the lack of paired data, can models be trained to generate sentences that reliably reflect the styles of different authors? RQ2: How can detector models and XAI techniques provide meaningful evaluation and explanation of stylistic generation? In addressing these questions, our contributions are threefold. First, we propose a framework for stylistic text generation without paired data, relying on single-token prompts as minimal signals. Second, we show how transformer-based detectors combined with XAI methods can provide reliable evaluation and interpretable insights into stylistic differences. Third, we show how these methods can support analysis, preservation, and creative exploration of literary style at scale. II Related Work Research on style transfer and stylistic text generation has followed several complementary directions. Early work focused on generative models that modify sentence style based on categorical attributes. For example, Logeswaran et al. [ 24 ] proposed a framework for content-preserving style transfer, while [ 8 ] fine-tuned GPT-2 on the GYAFC corpus to transform informal text into formal sentences. Structured approaches, such as attentional auto-encoders combined with style classifiers [ 38 ] , introduced mechanisms to maintain syntactic consistency by minimizing differences in part-of-speech structures between input and output sentences. This early stream of research established the foundation for later methods that increasingly sought to balance content fidelity with stylistic variation. Other studies have relied on adversarial learning. Lai et al. [ 19 ] introduced a word-level conditional GAN with a two-phase training scheme, using dual discriminators to balance content preservation and style transfer in non-parallel text settings. Lee [ 20 ] proposed the Stable Style Transformer, a delete-and-generate framework designed to improve robustness across evaluation metrics on non-parallel datasets. Variational autoencoder extensions such as [ 16 ] explored controllable text generation by disentangling latent content and style variables. GAN-based architectures like CTERM-GAN [ 1 ] incorporated syntactic and semantic constraints to enhance stylistic fidelity in controlled generation. More recently, diffusion-based approaches have emerged: Diffusion-LM [ 21 ] showed strong results in controllable text generation, and subsequent extensions such as fine-grained control via diffusion models [ 26 ] have further improved performance in low-resource style transfer scenarios. Together, these lines of work illustrate the gradual shift from discrete, attribute-driven models toward more flexible and powerful generative frameworks. Alongside advances in generation, efficient fine-tuning methods have gained attention. Low-Rank Adaptation (LoRA) [ 15 ] freezes the pre-trained model weights and injects trainable low-rank matrices into each transformer layer, reducing the number of trainable parameters while introducing no additional inference cost. Quantized LoRA (QLoRA) [ 9 ] combines low-rank adaptation with 4-bit quantization, enabling fine-tuning of very large language models on a single GPU with minimal loss in accuracy. Decomposed Rank-One Adaptation (DoRA) [ 23 ] extends this line of work by separating weight updates into magnitude and direction, which improves fine-tuning stability and efficiency without adding inference overhead. These developments show how advances in parameter-efficient training make it possible to experiment with stylistic generation even under resource constraints. Explainable AI (XAI) has become central to evaluating and interpreting modern language models. Broadly, methods can be divided into transparent

Read the complete text