QA-prompting: Improving Summarization with Large Language Models using Question-Answering
Neelabh Sinha · 2025
Read QA-prompting: Improving Summarization with Large Language Models using Question-Answering by Neelabh Sinha (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14347.txt.
Introduction
Neelabh Sinha proposes a simple trick to make language models summarize long texts better: answer questions first. Models often miss key facts buried in the middle of long inputs because of positional bias. Fine-tuning or multi-step pipelines can help but add cost and complexity. QA-prompting instead asks the model targeted questions about the text, then writes the summary enriched by those answers, all in one call per task.
Across several data sets and ten leading pre-trained models, the method beat plain baselines and prior approaches, lifting standard summary scores by up to twenty-nine percent. The questions must fit the domain: choosing them well was part of the method's success. Sinha presents the technique as a cheap, scalable fix that needs no retraining. It suggests that part of summarization failure is a retrieval failure, which a few good questions can cure.
Catalog record
| CREATOR | Neelabh Sinha |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-14347
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
QA-prompting: Improving Summarization with Large Language Models using Question-Answering Neelabh Sinha Abstract Language Models (LMs) have revolutionized natural language processing, enabling high-quality text generation through prompting and in-context learning. However, models often struggle with long-context summarization due to positional biases, leading to suboptimal extraction of critical information. There are techniques to improve this with fine-tuning, pipelining, or using complex techniques, which have their own challenges. To solve these challenges, we propose QA-prompting -- a simple prompting method for summarization that utilizes question-answering as an intermediate step prior to summary generation. Our method extracts key information and enriches the context of text to mitigate positional biases and improve summarization in a single LM call per task without requiring fine-tuning or pipelining. Experiments on multiple datasets belonging to different domains using ten state-of-the-art pre-trained models demonstrate that QA-prompting outperforms baseline and other state-of-the-art methods, achieving up to 29% improvement in ROUGE scores. This provides an effective and scalable solution for summarization and highlights the importance of domain-specific question selection for optimal performance 1 1 1 GitHub repository link of the implementation: https://github.com/neelabhsinha/qa-prompting . 1 Introduction Language Models (LMs) have revolutionized the application of Natural Language Processing. With instruction tuning ( Ouyang et al., 2022 ) , prompting ( Brown et al., 2020 ) , and in-context learning ( Wei et al., 2022a ; Dong et al., 2023 ) , LLMs perform well in most of the conditional generation tasks out-of-the-box. Specifically, in abstractive summarization, this approach yields highly fluent, consistent, and relevant summaries ( Tanya Goyal, 2022 ) that are even preferred over summaries generated by state-of-the-art (SOTA) models like BRIO ( Liu et al., 2022 ) . Figure 1: Framework for QA-prompting: Relevant questions are extracted from a corpus based on the domain of article, and a user defined top- k value. A prompt is constructed to first answer the questions, followed by generating summary. Despite the metrics rating the summaries low, several works argue that LM summaries are of high quality ( Zhang et al., 2023b ; Sinha et al., 2025a ) , fluent, grammatically correct, and largely relevant. But where facts are important, key information is sometimes not present. This problem proves to be more pronounced in the summarization of long context tasks. One of the factors contributing to this is the positional bias in LMs, where more recent tokens play a more critical role in the generation of the next token ( Sun et al., 2021 ) . It is also established that summaries are more faithful to tokens at the start and the end ( Wan et al., 2025 ) , ignoring the text in the middle. This phenomenon may be more pronounced in small to medium size LMs ( AI@Meta, 2024 ; Team et al., 2024 ; Jiang et al., 2023 ) because they don’t have the scale to extract deep contextual information effectively, leading to the generation of sub-optimal summaries. One way to handle this is by adapting LMs to the task via weight updates (fine-tuning), but this poses massive compute and data requirements, and does not generalize to all use cases. One can also do complex pipelining, or iterative refinement ( Zhang et al., 2023a ) , but these bring additional complications and inefficiencies. However, generating fluent, factually correct, contextually rich summaries efficiently is important to support generalization and scalability. This poses the following key challenges: ( C1 ) How to mitigate the positional bias of LM to generate a good summary? ( C2 ) How to make this approach generalize so that it can work with the summarization of all types of text? ( C3 ) How to achieve this with resource efficiency? To solve these challenges, we propose QA-Prompting – an approach to summarization by using question-answering and in-context learning with a single LM call. First, we show that vanilla prompting or in-context learning (ICL) generates sub-optimal summaries of articles. Then, using a manually crafted, potentially relevant set of questions, we identify top k questions that can aid summarization. Question answering is different from summarization, as it tries to focus on a specific part of the article rather than comprehending its global context at once. We use this property to extract relevant content from the article. Then, using the article and these questions, we prompt the LM to first generate answers to the questions, followed by generating a summary. By trying to answer the relevant questions first, the LM extracts useful information and keeps it in recent context, which further helps it to generate good summaries (addressing C1). It also filters out noise from the long context article that is not relevant for the summary. Using small LMs in the 0.5B-12B range, we show that this approach significantly improves the quality of summaries (addressing C2). QA-prompting uses a single LM call for a task and works with a pre-trained model without modifications (addressing C3). We also conduct a detailed ablation study to validate our design choices. Our questions are domain-specific, i.e., they are different for tasks belonging to different domains (e.g., news and research). Identifying top k questions is an overhead, but needs to be done only once for a domain. We show that this domain-level adoption of QA-prompting is necessary and performs better than keeping a standard set for all tasks. The key contributions of this work is: • QA-prompting – a novel domain-adaptive QA-driven prompting for text summarization to improve the extraction of critical information. • Leveraging question-answering as an intermediate step to generate summaries that are both contextually rich and factually accurate in a single LM call. • Demonstrating its effectiveness with pre-trained LMs in 0.5B-12B, achieving 9-30\% improvement in ROUGE scores. 2 Related Work Summarization using LLMs : LLMs have produced state-of-the-art results in abstractive summarization and have significantly accelerated the research in this area ( Pu et al., 2023 ) . Their summaries have been widely accepted through various works ( Tanya Goyal, 2022 ; Zhang et al., 2023b ) . SummIt ( Zhang et al., 2023a ) iteratively prompts ChatGPT to generate a summary and keep improving it based on feedback from an evaluator LLM. But, it is inefficient and costly due to multiple LM calls. Chain-of-density ( Adams et al., 2023 ) gradually improves summaries using GPT-4 by iteratively adding more content while keeping the length constant. This is also inefficient and makes the summary less readable. Positional bias in LLMs : In transformer models, earlier and recent tokens are known to dominate the prediction of the next token ( Wan et al., 2025 ; Sun et al., 2021 ) . Streaming LLM ( Xiao et al., 2023 ) was able to generate high-quality text, using just an attention sink and a local attention window. Local context also dominates LM performance in multiple choice QA ( Zheng et al., 2024 ; Pezeshkpour and Hruschka, 2023 ) and arithmetic tasks ( Shen et al., 2023 ) . Prompting and in-context learning : Prompting emerged as an effective way of utilizing LMs without fine-tuning ( Radford et al., 2019 ) . GPT-3 ( Brown et al., 2020 ) introduced in-context learning (ICL), showing that giving LMs some pre-engineered examples helps them to understand the task better and