Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs
Eilam Cohen Itamar Bul Danielle Inbar Omri Loewenbach, [email protected], [email protected], [email protected] · 2026
Read Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs by Eilam Cohen Itamar Bul Danielle Inbar Omri Loewenbach, [email protected], [email protected], [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05794.txt.
Introduction
Eilam Cohen and 3 co-authors compare in 2026 two ways of making language models simplify text: careful prompting versus fine-tuning. Their study, called Simplify-This, uses encoder-decoder models across several benchmarks and many evaluation metrics. Text simplification rewrites passages in plainer language while keeping meaning, a practical aid for many readers.
Fine-tuned models consistently produced stronger structural simplification, actually changing sentence structure rather than echoing the input. Prompted models often scored higher on semantic similarity, but partly because they tended to copy their inputs. Human evaluators favored the fine-tuned outputs overall. The authors release code, a cleaned dataset, model checkpoints, and prompt templates so others can reproduce and extend the comparison.
Catalog record
| CREATOR | Eilam Cohen Itamar Bul Danielle Inbar Omri Loewenbach, [email protected], [email protected], [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-05794
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Simplify-This: A Comparative Analysis of Prompt-Based and Fine-Tuned LLMs Eilam Cohen Itamar Bul Danielle Inbar Omri Loewenbach, [email protected], [email protected], [email protected] Abstract Large language models (LLMs) enable strong text generation, and in general there is a practical trade-off between fine-tuning and prompt engineering . We introduce Simplify-This , a comparative study evaluating both paradigms for text simplification with encoder–decoder LLMs across multiple benchmarks, using a range of evaluation metrics. Fine-tuned models consistently deliver stronger structural simplification , whereas prompting often attains higher semantic similarity scores yet tends to copy inputs . A human evaluation favors fine-tuned outputs overall. We release code, a cleaned derivative dataset used in our study, checkpoints of fine-tuned models, and prompt templates to facilitate reproducibility and future work. 1 Introduction Large language models (LLMs) have rapidly transformed natural language processing (NLP). Two major paradigms dominate contemporary use: fine-tuning , where models are adapted to specific tasks via supervised training on annotated corpora, and prompt engineering , where task performance is elicited through carefully designed natural-language instructions without modifying model parameters. Fine-tuning has traditionally yielded strong in-domain results, but it requires substantial computational resources and curated datasets. Prompting, by contrast, offers flexibility and low cost, yet its effectiveness varies widely across tasks and models. Among NLP applications, text simplification occupies a distinctive role. The task aims to reduce the linguistic complexity of a sentence while preserving its meaning ( 27 ) . Unlike summarization, which compresses content, simplification targets accessibility by rewriting text to be more comprehensible for non-native speakers, children, or individuals with reading difficulties. Despite its apparent simplicity, achieving high-quality simplification is challenging: models must balance readability, fluency, and semantic faithfulness. Classical approaches often struggle to avoid trivial copying or excessive distortion, while automatic metrics capture complementary yet incomplete aspects of quality (e.g., structural, readability, and semantic similarity measures). Foundational advances in pre-trained sequence-to-sequence models such as T5 ( 25 ) and BART ( 17 ) have significantly improved the state of the art in text generation, and subsequent work has applied these architectures to simplification with encouraging results ( 19 ; 5 ; 14 ) . Instruction-tuned variants such as Flan-T5 ( 7 ) further raise the question of whether prompting alone can rival fine-tuning in this task. However, systematic comparisons between the two paradigms in text simplification remain limited. In this paper, we present Simplify-This , a comparative study of fine-tuning versus prompt-based approaches for text simplification. We evaluated a diverse set of pre-trained models across three commonly used datasets: ASSET ( 1 ) , Med-EASi ( 5 ) , and OneStopEnglish ( 30 ) . Our analysis integrates automatic metrics with human preference judgments, providing a comprehensive view of the trade-offs between edit-sensitive and faithfulness-oriented criteria. Beyond reporting results, we highlight when prompting can approximate fine-tuning, where fine-tuning remains indispensable, and how dataset domain may influence the relative success of each paradigm. Overall, our contributions are threefold: • We conduct, to the best of our knowledge, among the first systematic head-to-head comparison of fine-tuning and prompting for text simplification across multiple datasets, models, and evaluation methods. • We release cleaned data ( WikiLarge-Clean ), code, prompts, and fine-tuned checkpoints to support reproducibility and further research. • We provide both quantitative and qualitative analyses, including human evaluation, to contextualize the strengths and limitations of each approach. All resources are publicly available (see Section 6 ). 2 Related Work 2.1 Text simplification with LLMs The emergence of instruction-tuned LLMs has reshaped English text simplification (TS). The BLESS benchmark systematically evaluated 44 off-the-shelf LLMs under few-shot prompting across Wikipedia, news, and medical domains, showing that the best prompted LLMs can perform comparably to state-of-the-art supervised TS baselines without task-specific fine-tuning ( 14 ) . Building on this, 35 conducted an error-based human evaluation and found that GPT-4 produces fewer erroneous simplifications than prior fine-tuned models, though challenges remain in lexical paraphrasing and content fidelity. They further caution that commonly used automatic metrics can be insensitive among high-quality outputs. 2.2 Prompting vs. fine-tuning Across related rewriting tasks, comparisons between prompting and fine-tuning reveal a pragmatic trade-off. In the BioLaySumm shared task, fine-tuning a Longformer-based model outperformed GPT-4 prompting when the full training set was available, whereas GPT-4 prompting excelled in low-data settings ( 29 ) . This suggests prompting is often effective in scarce-data regimes, while fine-tuning often excels when large, domain-specific corpora exist. For document-level TS, 9 propose a progressive multi-stage prompting method (ProgDS), which decomposes simplification into discourse-, topic-, and lexical-level phases. ProgDS significantly outperforms direct one-shot prompting and smaller fine-tuned models, underscoring the potential of structured prompt design when simplifying long texts. 2.3 Datasets and domains Prior work on simplification commonly evaluates models on ASSET ( 1 ) , Newsela ( 36 ) , and Med-EASi ( 5 ) . The BLESS benchmark explicitly included these domains and reported that strong LLMs generalize across them, though with domain-specific error patterns ( 14 ) . In addition, many studies rely on the WikiLarge corpus ( 40 ; 37 ) , though several limitations have been documented ( 1 ) . 2.4 Metrics and control The field routinely reports SARI ( 37 ) , FKGL ( 15 ) , BERTScore, ( 39 ) and LENS ( 18 ) . 35 emphasize that automatic metrics are sometimes insensitive among high-quality outputs, motivating human evaluation as a complementary measure. Beyond generic metrics, readability-controlled rewriting remains challenging: 10 show that LLMs can be prompted to adjust complexity up or down but often fail to reach precise grade levels, with readability still correlating strongly with the original input. 3 Methodology 3.1 Task We study sentence-level text simplification: given an input sentence x, generate an output y that preserves meaning and is easier to understand while improving readability. 3.2 Models For this study, we focused on sequence-to-sequence (seq2seq) architectures , as they are well-suited to natural language generation tasks where the goal is to transform an input sequence into a modified but semantically related output. Seq2seq models have proven highly effective in a wide range of NLP applications, including machine translation ( 4 ) , summarization ( 26 ) , and dialogue systems ( 31 ) . Their encoder–decoder structure allows the model to capture the semantic content of an input sentence and generate a controlled output, making them especially appropriate for text simplification , where meaning must be preserved while surface form is simplified ( 40 ; 37 ) . In practice, one notable example is MUSS , a BART-based model fine-tuned specifically for text simplification ( 19 ) . MUSS demonstrated that pretrained seq2seq models, when adapted through targeted fine-tuning, can outperform previous supervised approaches and achieve state-of-the-art results on benchmarks such as WikiLarge Our project was also constrained by computational and financial resources. Training large-scale models is prohibitive in terms of cost and compute time. To ensure we could conduct full fine-tuning experiments within our limited budget, we deliberately selected open-source models under 1 billion parameters from Hugging Face . This allowed us to run multiple experiments efficiently on limited hardware while still comparing competitive and diverse architectures. We therefore selected a representative set of open-source encoder–decoder models from the Hugging Face Hub ( 34 ) . Our