Long text outline generation: Chinese text outline based on unsupervised framework and large language model
Yan Yan, Yuanchi Ma · 2024
Read Long text outline generation: Chinese text outline based on unsupervised framework and large language model by Yan Yan, Yuanchi Ma (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-00810.txt.
Introduction
Yan Yan and Yuanchi Ma draft outlines for very long Chinese texts, such as novels, with help from large language models. Their 2024 paper notes that past tools cope with short and mid-length texts but lose the thread in long fiction. Chapter cuts come out muddled and outlines hard to read. Their method first builds a graph for each chapter from named things and sentence links.
A graph-based module turns those chapter graphs into compact codes. A chain-based step then finds where the plot breaks into parts. Last, a large model sums up each part and assembles the full outline. On tests of plot cuts and outline ease, the method beat deep models and large models used alone.
Catalog record
| CREATOR | Yan Yan, Yuanchi Ma |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-00810
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Long text outline generation: Chinese text outline based on unsupervised framework and large language model Yan Yan, Yuanchi Ma Abstract Outline generation aims to reveal the internal structure of a document by identifying underlying chapter relationships and generating corresponding chapter summaries. Although existing deep learning methods and large models perform well on small- and medium-sized texts, they struggle to produce readable outlines for very long texts (such as fictional works), often failing to segment chapters coherently. In this paper, we propose a novel outline generation method for Chinese, combining an unsupervised framework with large models. Specifically, the method first generates chapter feature graph data based on entity and syntactic dependency relationships. Then, a representation module based on graph attention layers learns deep embeddings of the chapter graph data. Using these chapter embeddings, we design an operator based on Markov chain principles to segment plot boundaries. Finally, we employ a large model to generate summaries of each plot segment and produce the overall outline. We evaluate our model based on segmentation accuracy and outline readability, and our performance outperforms several deep learning models and large models in comparative evaluations. 1 Introduction Well-written stories are often composed of numerous semantically coherent chapters, with each chapter or group of chapters centered around a specific theme. An outline can concisely capture the content structure of a document, providing clear guidance for navigation and significantly reducing the cognitive burden of understanding the entire text. Furthermore, it helps uncover the underlying thematic structure of the text. Outline generation captures various thematic elements of a text, including subtitles, plot points, and other key aspects of the narrative. Additionally, outlines facilitate a wide range of text analysis applications. They are not only beneficial for traditional downstream NLP tasks, such as document summarization Xiao and Carenini 2019 and discourse parsing Huber et al. 2022 , but also play a crucial role in large language models (LLMs). For example, during retrieval-augmented generation (RAG) in large language models Shi et al. 2024 , it is essential to extract the necessary information from long documents. The paragraph-level thematic structure of a document can aid in quickly locating the approximate position of the required content within a lengthy text, thereby reducing the search space. The relevant process concept is outlined as follows. Figure 1: Multimodal nonparametric clustering algorithm via Monte Carlo method and variational autoencoder diagram. Previous work on outline generation Zhang et al. 2019 ; Zhou et al. 2015a has primarily focused on short- and medium-length texts, such as news articles and announcements, helping readers quickly grasp the structure of the content. The vertical domains involved include sociology, psychology and economics Ma et al. 2023 . These methods typically generate outlines based on natural paragraphs. However, for fictional works such as Game of Thrones (GoT), the Marvel Comics Universe (MCU), Greek mythology, or epic novels like Leo Tolstoy’s War and Peace or Don Winslow’s The Cartel, the task of outline generation is much more challenging due to their length and intricate semantic structures Hertling and Paulheim 2020 . While novels are generally meant for entertainment, some of these works also reflect subcultural trends and capture the zeitgeist of particular eras. Analyzing their narrative structures and networks is of great interest to scholars in the humanities. For instance, War and Peace is set against the backdrop of Napoleon’s wars in Russia, while The Cartel trilogy intertwines both fact and fiction concerning drug trafficking. Therefore, generating outlines for such long texts not only helps general readers quickly understand the relevant plots but also provides a valuable data foundation for historians, social scientists, media psychologists, and cultural studies scholars to conduct deeper analyses of these complex works Chu et al. 2021 . Long fictional texts are composed of significant plot points, where the focus is less on basic relationships such as birthplaces or spouses, and more on specific events like alliances, betrayals, killings, or clan conflicts. These fictional works often have large fan communities, and search engines frequently receive queries like "When does Xiao Yan obtain the Bone Spirit Cold Flame?" (Battle Through the Heavens) Hua et al. 2016b . For texts spanning millions of words, it is evidently challenging to locate a particular plot without an outline. Therefore, it is necessary to develop an outline generation method tailored to long fictional texts to address this problem. Upon further examination of the outline generation challenge, we observe that it actually involves two structured prediction tasks: (1) identifying chapter features and plot boundaries, and (2) generating chapter summary titles. These two tasks correspond to predicting the hierarchical relationship between chapters and summarizing individual chapters. While the second task can be well-handled by existing large language models (LLMs), particularly for short- and medium-length texts, LLMs often exhibit inaccuracies and increased hallucination issues when applied to longer texts. For instance, when tasked with summarizing a work of over a million words, LLMs may overlook key plot points, preventing readers from fully grasping the narrative [add experiment]. Therefore, we consider whether an ideal outline generation framework could extend the strengths of large models to ultra-long texts. The key challenge here lies in accurately identifying the sequence and features of chapters to obtain precise plot boundaries. In our work, we propose a new end-to-end architecture to address this challenge. The key idea is to enhance large language model (LLM) outputs by guiding them with enriched information, specifically by determining plot boundaries through a neural network before using them to guide the LLM in generating a detailed outline. We posit that graph data can better represent relationships between entities within chapters, thus reflecting chapter characteristics more effectively. Therefore, our method first generates entity nodes through a chapter-level graph data generation module, followed by constructing the adjacency matrix between nodes based on syntactic dependency relationships. For node feature vectors, we not only select entity word vectors but also expand the feature set to include the tf-idf matrix of the entities, and we incorporate chapter numbers to represent contextual coherence. We then apply an improved graph neural network (GNN) based on graph attention layers (GAT) to learn from the chapter graph data. To this, we add a convolutional module for feature extraction and dimensionality reduction of deep chapter graph embeddings. For each chapter embedding, we perform chain-based prediction: specifically, we determine significant plot points and boundaries using Markov chains and path dependence based on their potential distances in feature space. Finally, LLMs are used to generate the themes and summaries for each plot segment, resulting in the final outline. To facilitate our research, we constructed a new benchmark dataset in Chinese. As we observed, ultra-long texts in the domain of fictional literature often consist of millions of words. In this dataset, we not only provide the original literary works but also manually generated outlines to serve as a reference for evaluating experimental performance. For assessment, we compared several state-of-the-art methods to verify the effectiveness of our model, including rule-based models and large language models. The experimental results demonstrate that our proposed method significantly outperforms all baselines. We also conducted a detailed analysis of the proposed model, including an analysis of the readability of the generated outlines, to better understand the learned content structure.