Leveraging Large Language Models for Zero-shot Lay Summarisation in Biomedicine and Beyond
Tomas Goldsack, Carolina Scarton, Chenghua Lin · 2025
Read Leveraging Large Language Models for Zero-shot Lay Summarisation in Biomedicine and Beyond by Tomas Goldsack, Carolina Scarton, Chenghua Lin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-05224.txt.
Introduction
Tomas Goldsack, Carolina Scarton, and Chenghua Lin test language models as plain-language summarizers in a 2025 paper. Research papers in biomedicine are hard for non-experts to read. A lay summary restates the work in everyday words, and models could draft them without task-specific training.
Their two-stage method mirrors how humans write such summaries, and human judges preferred its output more as models grew larger. They also tested models as judges of summaries and found their choices matched human panels well. Early trials on computing papers show the skill transfers to a new field. The work sets out early best practice for zero-shot lay summary.
Catalog record
| CREATOR | Tomas Goldsack, Carolina Scarton, Chenghua Lin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-05224
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Leveraging Large Language Models for Zero-shot Lay Summarisation in Biomedicine and Beyond Tomas Goldsack, Carolina Scarton, Chenghua Lin Abstract In this work, we explore the application of Large Language Models to zero-shot Lay Summarisation. We propose a novel two-stage framework for Lay Summarisation based on real-life processes, and find that summaries generated with this method are increasingly preferred by human judges for larger models. To help establish best practices for employing LLMs in zero-shot settings, we also assess the ability of LLMs as judges, finding that they are able to replicate the preferences of human judges. Finally, we take the initial steps towards Lay Summarisation for Natural Language Processing (NLP) articles, finding that LLMs are able to generalise to this new domain, and further highlighting the greater utility of summaries generated by our proposed approach via an in-depth human evaluation. 1 Introduction The goal of Lay Summarisation is to create a summary that effectively communicates the key concepts and findings of a technical article to a non-expert audience ( Goldsack et al. 2022 ; Zhang et al. 2024 ) . To achieve this, a well-written lay summary must, without hype or exaggeration, convey these findings clearly and concisely using everyday language, ensuring that it remains accessible to readers without specialised knowledge ( King et al. 2017 ) . In recent years, the task of automatic Lay Summarisation has attracted increased interest in the research community, driven by the need to broaden access to scientific research. However, due to a lack of availability of publically accessible reference data, research for this task has been restricted to a few select domains, with Biomedicine being the most prominent. Figure 1: A Visualisation of our two-stage Lay Summarisation framework, based on the real-life process of the eLife Journal. In recent years, the rise of Large Language Models (LLMs) has drastically altered the research landscape for Natural Language Generation (NLG), including for tasks such as Summarisation ( Goyal et al. 2022 ; Goldsack et al. 2023b ) . For Lay Summarisation specifically, the results of the recent BioLaySumm 2023 shared task suggest that LLMs have the potential to also push the boundaries of task performance ( Goldsack et al. 2023a ) , with two of the top three submissions adopting them within their approach ( Turbitt et al. 2023 ; Sim et al. 2023 ) . At the forefront of this changing landscape are current LLMs’ significantly improved zero-shot capabilities, emerging from the increased scale of models ( Kaplan et al. 2020 ) and breakthrough techniques such as instruction-tuning ( Wei et al. 2022 ) . Importantly, these zero-shot capabilities have significantly widened the scope of LLM applications, enabling us to explore tasks that were previously impossible due to a lack of training data. For Lay Summarisation, these models now provide the opportunity to explore domains outside of Biomedicine and improve accessibility to research in all domains. However, little is known about how best to utlise LLMs to generate lay summaries in a zero-shot setting, or how best to evaluate generated summaries in the absence of references. In this work, we attempt to address these questions and lay the groundwork for future Lay Summarisation research in unexplored domains. We propose a novel two-stage prompting framework for zero-shot Lay Summary generation inspired by real-world practices, before thoroughly evaluating the performance of our method in two domains: Biomedicine and Natural Langauge Processing (NLP). Our results show that summaries generated using the proposed framework are increasingly preferred by human judges as we increase the size of the underlying LLM, and that a panel of LLMs proves an effective proxy to human judges. Furthermore, we find that LLMs can effectively generalise to the previously-unexplored domain of NLP articles, and that the summaries generated with our proposed framework provide lay readers with a more well-rounded understanding of article contents in this domain. 2 Task Decomposition Framework We explore the use of a two-stage framework for Lay Summarisation, based on the practices of the journal eLife. In a recent study, Goldsack et al. 2022 found that the lay summaries from eLife, crafted internally by journal editors and hired writers in collaboration with article authors, were notably more accessible than the author-written lay summaries from PLOS (the Public Library of Science). 1 1 1 eLife lay summaries are on average longer, more readable, and more abstractive than those of PLOS, positively impacting their comprehensibility. To produce their lay summaries, eLife adopts a multi-stage process involving different actors. King et al. 2017 describe this process, whereby the paper authors are asked to answer a set of questions concerning the motivation and impact of their work, aiming to extract information that may not be explicit with the article itself but is essential to lay audience understanding. Upon inquiry with the team at eLife, we were provided with these questions, which inform our approach. As visualised in Figure 1 , our framework is based on the introduction of an additional “question-answering" stage in the summary generation process, with the answers generated from this stage used as an additional data source during summary generation. By introducing this additional stage, we aim to simulate the process that produces the high-quality lay summaries of eLife, and make explicit the important questions that should be addressed in the final summary. A detailed description of each stage is provided below: 1. Question-answering Given the article text, the LLM is asked to play the role of the paper author and answer four questions about the article. As previously mentioned, these questions are derived from those that eLife asks of authors Specifically, the questions are: 1) What background information would someone who is completely unfamiliar with your field need to know to understand the findings in your paper? 2) What exact research question did you set out to answer and why? 3) What are the most important findings of your paper? and 4) Who might eventually benefit from the findings of your study, and what would need to be done before we could achieve these benefits? 2 2 2 Alongside each question, additional guidance for answering is also presented. We provide a full explanation of this, and the prompts used for each agent, in Appendix A . 2. Summary generation Given the article text and the “author’s" answers from the previous stage, we instruct the LLM to play the role of a writer and generate the lay summary based on guidelines derived from the analyses of Goldsack et al. 2022 . These guidelines describe the desirable length, language, and structure of the generated summary. Model # Params QA Relevance Readability Factuality H2H R-1 \uparrow R-2 \uparrow R-L \uparrow BeS \uparrow FKGL \downarrow DCRS \downarrow BaS \uparrow PoLL \uparrow PoH \uparrow Phi2 2.7B ✗ 41.66 9.10 39.39 83.27 13.08 9.05 -3.06 0.65 0.80 ✓ 35.06 7.97 32.51 83.30 12.61 9.62 -3.77 0.35 0.20 Mistral 7B ✗ 44.02 10.39 41.26 84.15 14.08 10.42 -3.60 0.60 0.75 ✓ 44.60 9.95 41.77 84.29 14.17 10.19 -3.58 0.40 0.25 Mixtral 46B ✗ 45.59 11.02 42.85 84.17 14.08 9.69 -3.06 0.40 0.45 ✓ 45.49 10.41 42.90 83.95 13.92 9.57 -3.12 0.60 0.55 LLAMA3 70B ✗ 45.59 11.22 43.03 84.72 10.37 8.43 -3.34 0.20 0.35 ✓ 44.90 10.68 41.93