Progressive Document-level Text Simplification via Large Language Models

Dengzhao Fang, Jipeng Qiang, Yi Zhu, Yunhao Yuan, Wei Li, Yan Liu · 2025

Read Progressive Document-level Text Simplification via Large Language Models by Dengzhao Fang, Jipeng Qiang, Yi Zhu, Yunhao Yuan, Wei Li, Yan Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-03857.txt.

Introduction

Dengzhao Fang and co-authors simplify whole documents with large language models, stage by stage. Most past work eased single words or sentences, while long-document simplification stayed underexplored. Big models often treat the task as mere summarizing, which loses content. A proper simplification must keep the document whole while easing its discourse, sentences, and words in a balanced way.

Human editors simplify in a hierarchy, and the proposed method, ProgDS, copies that strategy. It splits the job into discourse-level, topic-level, and word-level passes, with models collaborating across the stages. In experiments, the staged method beat both smaller dedicated models and direct prompting of large models. The authors say the result advances the state of the art for document simplification. The work points to decomposition as a general trick for long-text tasks where a single pass tends to drift.

Catalog record

CREATORDengzhao Fang, Jipeng Qiang, Yi Zhu, Yunhao Yuan, Wei Li, Yan Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-03857

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Progressive Document-level Text Simplification via Large Language Models Dengzhao Fang, Jipeng Qiang, Yi Zhu, Yunhao Yuan, Wei Li, Yan Liu Abstract Research on text simplification has primarily focused on lexical and sentence-level changes. Long document-level simplification (DS) is still relatively unexplored. Large Language Models (LLMs), like ChatGPT, have excelled in many natural language processing tasks. However, their performance on DS tasks is unsatisfactory, as they often treat DS as merely document summarization. For the DS task, the generated long sequences not only must maintain consistency with the original document throughout, but complete moderate simplification operations encompassing discourses, sentences, and word-level simplifications. Human editors employ a hierarchical complexity simplification strategy to simplify documents. This study delves into simulating this strategy through the utilization of a multi-stage collaboration using LLMs. We propose a progressive simplification method (ProgDS) by hierarchically decomposing the task, including the discourse-level, topic-level, and lexical-level simplification. Experimental results demonstrate that ProgDS significantly outperforms existing smaller models or direct prompting with LLMs, advancing the state-of-the-art in the document simplification task. keywords Document Simplification, Large Language Model, ChatGPT, Multi-stage System, Text Generation 1 Introduction Text simplification aims to simplify the input text by reducing its complexity to make it more understandable for a wider audience, including non-native speakers ( Paetzold and Specia, 2016 ) and individuals with cognitive impairments ( Gooding, 2022 ; Kajiwara et al., 2013 ; Paetzold, 2015 ) . Lexical-level and sentence-level simplification have been the main focus in the field of text simplification by training neural network models or fine-tuning pre-trained language models using supervised data ( Paetzold and Specia, 2016 ; Gooding, 2022 ) . Since the challenge of generating long sequential text output has persisted, document-level simplification (DS) has attracted little attention. Traditional approaches for document simplification often rely on simplifying sentences individually, without considering the broader context of the document. This disregard for the overall document context can lead to a loss of coherence and integrity in the simplified version. Recently, these DS work ( Cripwell et al., 2023 ; Sun et al., 2023 ) primarily focused on sentence-level simplification operations (copy, rephrase, split, or delete). Binova et al. ( Blinova et al., 2023 ) proposed a two-stage framework, which first generated one summary of the input document and generated the simplified summary. The recent development of Large Language Models (LLMs), such as ChatGPT, has ushered in a new paradigm in natural language processing ( Zhou et al., 2023 ) . Several studies ( Feng et al., 2023 ; Kew et al., 2023 ) have highlighted the significant potential of Large Language Models (LLMs) in sentence simplification, requiring no specialized training data or modifications to model parameters. Table 1 : Comparison of token counts between manually generated simplified documents and those generated by ChatGPT on Newsela. Newsela-A and Newsela-B are composed of 500 sample pairs from the Newsela dataset with article lengths below 1000 tokens and with article lengths exceeding 1,000 tokens. For each original article (SRC), three simplified versions (RE1, RE2, and RE3) by humans are shown here, where RE3 is the simplest. ’P1’, ’P2’, and ’IC’ represent three simplified versions generated by ChatGPT guided by three different prompt templates. Dataset SRC RE1(RE2, RE3) P1(P2,IC) Newsela-A 818 833 (805, 768) 352(334, 308) Newsela-B 1506 1286 (995, 863) 445(329, 378) We test ChatGPT on one DS data Newsela ( Xu et al., 2015 ) constructed by humans, where each article of Newsela was rewritten four times by professional editors for children at different grade levels. As shown in Table 1 , ChatGPT with three different prompts commonly produces a summary of the original document rather than a simplification of the document. The most obvious sign is that the length of the simplified output is much shorter than the references. Although we tried three different prompting strategies including direct prompting (P1), pointing out the difference between summarizing and simplifying (P2), and few-shot setting (IC), they still yielded unsatisfactory results. Why do LLMs work well for document summarization and not for DS tasks? DS has the following two challenges when applied to LLMs. (1) Content Preservation: While simplifying text, it is important to retain the essential information and meaning of the original document. But, while generating text in lengthy sequences, it is a formidable challenge to decide which details to preserve and which to simplify or omit to maintain the document’s overall integrity and message. (2) Ambiguity and Subjectivity: Simplification involves making decisions about how to convey information in a clearer and more straightforward manner. Large language models may struggle to consistently produce simplifications that meet human expectations and preferences, especially for longer texts where ambiguity and nuance are more prevalent. We see that LLMs fall short of achieving DS with a single prompt. LLM-based multi-stage systems have achieved considerable progress in complex problem-solving ( Guo et al., 2024 ) . This paper aims to explore the design of a multi-stage-based approach that can imitate human editors’ approach to simplifying long documents using LLMs. The human editor accomplishes the simplification of the document through the following steps: (1) Before simplifying the text, it’s crucial to understand the document’s purpose and identify the main topics or sections of the document. (2) Then, the content of each topic is simplified individually, including sentence rephrasing, deletion, lexical simplification, and other operations. Therefore, to simulate this simplified process of human editors, we propose a progressive simplification method (ProgDS) that enables LLMs to follow human-simplified operations under multi-stage collaboration. Specifically, ProgDS dissects the task of DS into three hierarchical steps: discourse-level, topic-level, and lexical-level simplification. Starting from the overall logical structure, then moving on to the arrangement and combination of paragraphs and sentences, and finally to lexical expressions, this task is approached gradually and systematically, simplifying each level step by step, rather than attempting to simplify all these levels simultaneously. In summary, the contributions of this paper are: (1) To our knowledge, we are the first to solve the task of long document-level simplification through the utilization of a multi-stage collaboration using LLMs. This bridges the gap between text simplification and practical applications with the assistance of LLMs. (2) We propose one novel DS method ProgDS by executing hierarchical complexity simplification. This alleviates the limitations of LLMs unable to simplify long documents through direct prompting. Our frameworks follow the simplification methods of human experts, integrating discourse-level, topic-level, and lexical-level simplification based on the principles of content and hierarchical division. (3) We evaluate ProgDS compared with existing methods on Wiki-auto and Newsela datasets. Experimental results show that ProgDS achieves state-of-the-art performance across various evaluation metrics. The benefits of our algorithm become more pronounced, especially when dealing with longer original documents. 2 Related Work 2.1 Document-Level Text Simplification The field of text simplification has traditionally focused on lexical and sentence-level methods ( Xu et al., 2015 ; Gooding, 2022 ) . While sequence-to-sequence frameworks are common in sentence simplification, they struggle with document simplification due to limited supervised data. Document simplification requires holistic consideration of semantic content, involving operations like sentence simplification, deletion, retention, and merging. Efforts to extract training corpora from sources like English Wikipedia (EW) and Simple English Wikipedia (SEW) face challenges due to suboptimal corpus quality. Recent advancements include the

Read the complete text