Can Good Writing Be Generative? Expert-Level AI Writing Emerges through Fine-Tuning on High-Quality Books Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Factors in Computing
Tuhin Chakrabarty, Paramveer S. Dhillon · 2026
Read Can Good Writing Be Generative? Expert-Level AI Writing Emerges through Fine-Tuning on High-Quality Books Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Factors in Computing by Tuhin Chakrabarty, Paramveer S. Dhillon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18353.txt.
Introduction
Tuhin Chakrabarty and Paramveer Dhillon ask whether a machine can learn to write at the level of a skilled human author. For a long time, fine prose was seen as a craft that needs a human voice and a human sense of style. New text systems can now copy the voice of many authors in a few seconds, at almost no cost in work.
Their study tests what comes from training such a system on books of high worth. The claim in the title is that strong, expert level prose can grow out of that kind of training. The work speaks to a live debate for writers and teachers. It also matters for firms that sell text tools. If style can be learned from good books, the line between a human draft and a model draft gets much harder to draw.
Catalog record
| CREATOR | Tuhin Chakrabarty, Paramveer S. Dhillon |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 105 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-18353
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can Good Writing Be Generative? Expert-Level AI Writing Emerges through Fine-Tuning on High-Quality Books Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain DOI: 10.1145/3772318.3791276 ISBN: 979-8-4007-2278-3/2026/04 Conference: Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain DOI: 10.1145/3772318.3791276 ISBN: 979-8-4007-2278-3/2026/04 CCS: Human-centered computing Empirical studies in HCI CCS: Human-centered computing Empirical studies in collaborative and social computing CCS: Computing methodologies Natural language generation Tuhin Chakrabarty, Paramveer S. Dhillon Abstract . Creative writing has long been considered a uniquely human endeavor, requiring voice and style that machines could not replicate. This assumption is challenged by Generative AI that can emulate thousands of author styles in seconds with negligible marginal labor. To understand this better, we conducted a behavioral experiment where 28 MFA writers (experts) competed against three LLMs in emulating 50 critically acclaimed authors. Based on blind pairwise comparisons by 28 expert judges and 131 lay judges, we find that experts preferred human writing in 82.7% of cases under the in-context prompting condition but this reversed to 62% preference for AI after fine-tuning on authors’ complete works. Lay judges, however, consistently preferred AI writing. Debrief interviews with expert writers revealed that their preference for AI writing triggered an identity crisis, eroding aesthetic confidence and questioning what constitutes “good writing.” These findings challenge discourse about AI’s creative limitations and raise fundamental questions about the future of creative labor. † † cc-license: by † † cc-license: by 1. Introduction In his book “The Program Era" ( 48 ) , eminent literary critic Mark McGurl discussed how " The rise of the creative-writing program stands as the most important event in postwar American literary history. " In these programs, "Show, don’t tell," which was a slogan in the nineteen-forties and fifties, shifted to an effectively opposite mantra, "Find your voice," that took over in the nineteen-sixties and seventies. McGurl observed that these guiding principles reflect evolving cultural beliefs—about identity, labor, gender, class, and notions of what counts as good writing —and that they have had a huge effect on the stories and novels that American writers have produced ( 49 ) . McGurl’s observation about what constitutes "good writing" takes on new significance in the age of generative artificial intelligence. The same qualities that creative writing programs cultivated—coherence, voice, narrative structure—are precisely what make their literary output valuable training data for LLMs ( 23 ) . Models trained on well-edited books and articles tend to generate more coherent and accurate outputs, something that’s crucial for establishing LLMs as intelligent systems. However extensive social experimentation with Generative AI has invited criticism on social media and in the popular news platforms that its writing has a disembodied “robovoice". Recent work from 12 shows how text generated from widely used LLMs are often rife with clichés, purple prose, poor sentence structure, and unnecessary exposition. AI writing often veers towards homogenization ( 19 ; 1 ) that is directly in contrast with what creative writing programs have taught. To counter this, some users and practitioners now prompt or train language models 1 1 1 https://community.openai.com/t/fine-tuning-the-model-for-writing-via-famous-author-style/512322 to emulate a specific writer’s style/voice ( 62 ) . This was also acknowledged by the US Copyright Office in their Generative AI Training report Part 1, where many commenters raised concerns about AI outputs imitating a creator’s style, which copyright does not protect as a separate element. Based on the U.S. Bureau of Labor Statistics (May 2023, national estimates), creative writing constitutes almost 50% of writing jobs that face significant risks from Generative AI-based automation. 2 2 2 https://lithub.com/against-ai-an-open-letter-from-writers-to-publishers/ Figure 1 . Three-phase study design showing (Phase 1) writing task with In-context prompting and fine-tuning approaches using authors with critically acclaimed style/voice. In the figure both writers and AI are trying to write a specific excerpt from My Year of Rest and Relaxation in the style/voice of Ottessa Moshfegh (Phase 2) Human-written vs AI-generated text evaluation task with rationales, and (Phase 3) Debrief session exploring sentiments and sensemaking when writers prefer AI over human writing. Figure illustrated by the first author of the paper Three-panel illustration of study design: Phase 1 shows a human writer and AI system both attempting to write in the style of author Ottessa Moshfegh, with the AI using either in-context prompting or fine-tuning on the author's works. Phase 2 shows an evaluator comparing human-written and AI-generated text side-by-side, providing preference judgments and written rationales. Phase 3 depicts a debrief interview session where participants discuss their reactions to learning AI was preferred over human writing. To understand how much “good writing" comes from having a distinct style/voice we conducted a behavioral study where 28 expert human writers were pitted against three LLMs (See Figure 1 ). We first preselected a group of 50 authors representing diverse cultural backgrounds and age groups, who are known for their critically acclaimed literary style/voice. We then assigned both human expert writers and LLMs the same task: to write a 200-450 word excerpt emulating the style and voice of a selected author, based on a writing prompt. For LLMs we included two conditions: (1) In-context Prompting, where models received the same prompt as human expert writers and (2) Fine-tuning, where models were additionally trained on each author’s complete oeuvre. There are several ways to get an LLM to emulate the style/voice of an author. In-context prompting puts representative examples in a prompt, whereas Fine-tuning actually modifies the model through training on author’s books and is more resource intensive. Following this, we recruited the same 28 writers along with 131 lay human participants from Prolific who acted as judges to evaluate these excerpts in a blind evaluation across two different axes i) Writing Quality ii) Stylistic Fidelity. We refer to writers as expert judges. Through planned pairwise contrasts between <Human\_written,AI\_generated> excerpts, both expert and lay judges provided their preferences as well as 2-5 sentence rationales grounded in snippets from these excerpts, explaining their preference. • RQ1 : What line of reasoning do experts and lay judges use to justify what qualifies as good writing? • RQ2 : After learning a preferred excerpt was written by AI, how do writers reconcile with others as well as their own judgments? • RQ3 : How does the discovery of choosing AI over human writing affect writers’ sense of professional identity as well as their fundamental understanding of what constitutes “good writing." • RQ4 : How do writers interpret AI’s advanced capabilities in terms of what they mean for creative writing programs and their writing career—and what changes do they suggest for educational institutions, publishing houses, and literary platforms? In addressing RQ1, we analyze the rationales collected alongside each blind preference. We find that lay judges primarily focus on surface-level qualities like flow, organization, and clarity, often citing whether writing is “easier to follow" or has emotional impact without analyzing the mechanisms behind it. Experts, in contrast, provide more detailed analysis focusing