Can one size fit all?: Measuring Failure in Multi-Document Summarization Domain Transfer
Alexandra DeLucia, Mark Dredze · 2025
Read Can one size fit all?: Measuring Failure in Multi-Document Summarization Domain Transfer by Alexandra DeLucia, Mark Dredze (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15768.txt.
Description
Alexandra DeLucia and Mark Dredze wrote a 2025 paper on when a summarization tool, trained on one kind of text, fails on another. The task is to summarize many documents at once, from news articles to transcripts of conversations. Training plans fall into four kinds: direct, chunk-first, extract-first, and a call to a large model. The team asks how and why a tool trained on one field fails when it summarizes another, with no fresh training.
They grade the tools on three fields and three marks: similarity to a reference summary, quality, and factual accuracy. A failure is defined as a drop in factual accuracy, a drift from the target, and a fall in quality. They also probe the standard grading tools themselves. A grading tool built for news can misgrade a summary of conversations, they warn. One size, the conclusion runs, does not fit all.
Catalog record
| CREATOR | Alexandra DeLucia, Mark Dredze |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-15768