Continually Self-Improving Language Models for Bariatric Surgery Question–Answering

Yash Kumar Atri, Thomas Shin, Thomas Hartvigsen · 2025

Read Continually Self-Improving Language Models for Bariatric Surgery Question–Answering by Yash Kumar Atri, Thomas Shin, Thomas Hartvigsen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16102.txt.

Introduction

Yash Kumar Atri, Thomas Shin, and Thomas Hartvigsen built this 2025 chatbot helper for weight-loss surgery care. Surgery for severe obesity works best when patients stay engaged with surgeons, dietitians, and others for years. Access barriers keep many from timely, expert-checked answers. Their system pairs a language model with live retrieval of medical evidence.

The tool, bRAGgen, refreshes its answers whenever its confidence dips below shifting thresholds. That self-updating design keeps replies current and lowers the risk of stale advice. The team also built bRAGq, a set of 1,302 surgery questions checked by an expert surgeon, the first large benchmark for this field. In two-stage testing, their system beat leading models on model-scored and surgeon-scored measures alike.

Catalog record

CREATORYash Kumar Atri, Thomas Shin, Thomas Hartvigsen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-16102

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Continually Self-Improving Language Models for Bariatric Surgery Question–Answering Yash Kumar Atri, Thomas Shin, Thomas Hartvigsen Abstract While bariatric and metabolic surgery (MBS) is considered the gold standard treatment for severe and morbid obesity, its therapeutic efficacy hinges upon active and longitudinal engagement with multidisciplinary providers, including surgeons, dietitians/nutritionists, psychologists, and endocrinologists. This engagement spans the entire patient journey, from preoperative preparation to long-term postoperative management. However, this process is often hindered by numerous healthcare disparities, such as logistical and access barriers, which impair easy patient access to timely, evidence-based, clinician-endorsed information. To address these gaps, we introduce bRAGgen , a novel adaptive RAG-based model that autonomously integrates real-time medical evidence when response confidence dips below dynamic thresholds. This self‑updating architecture ensures that responses remain current and accurate, reducing the risk of misinformation. Additionally, we introduce bRAGq , a curated dataset of 1,302 1,302 bariatric surgery–related questions, validated by expert bariatric surgeon, constituting the first large‑scale, domain‑specific benchmark for comprehensive MBS care. In a two-phase evaluation, bRAGgen is benchmarked against state-of-the-art models using both large language model (LLM)-based metrics and expert surgeon review. Across all evaluation dimensions, bRAGgen demonstrates substantially superior performance in generating clinically accurate and relevant responses. Data and Code available at https://github.com/yashkumaratri/bRAGgen 1 Introduction Severe obesity and its subsequent metabolic disease have become a widespread endemic condition, leading to a projected incidence of 25% across the United States by 2030, causing a massive health burden in the general US population ( Ward et al., 2019 ) . Metabolic and bariatric surgery (MBS) remains the gold standard treatment for severe obesity and metabolic disease, with over 270,000 annual procedures in the United States ( Clapp et al., 2024 ; Mechanick et al., 2020 ; Barres et al., 2013 ; Loos and Yeo, 2022 ; Setarehdan et al., 2023 ) . However, successful weight loss post-MBS heavily relies on patient education, which has led National Bariatric Surgery and Medical societies to emphasize extensive education services for MBS patients ( Mechanick et al., 2020 ) . Perioperative MBS patient education includes information about dietary modification, adjunctive lifestyle modifications, expectations surrounding postoperative complications, and psychosocial support; all important mitigators of periprocedural complications and postoperative weight regain, which can afflict up to 64% of patients in 5–10 years post-MBS ( Groller et al., 2017 ; Bjerkan et al., 2022 ; David et al., 2020 ; McLennan et al., 2023 ; Kim et al., 2023 ) . And after surgery, patients only attend their yearly followup an average of 6.5–29.6% of the time, leaving patients and providers with limited opportunities to communicate. Overall, the lack of sustained patient engagement and education post-MBS is a critical impediment to optimal postoperative outcomes with many causes, including low health literacy rates, information inaccessibility, and geographic distances to healthcare providers ( Setarehdan et al., 2023 ; Mechanick et al., 2020 ; Schlottmann et al., 2023 ; Bartholomay et al., 2024 ) . Given these challenges, there is a clear need for scalable, accessible, and continually updated educational and decision‑support tools tailored to the unique needs of MBS patients—spanning from preoperative preparation to long-term postoperative management ( P et al., 2024 ) . Traditional patient education materials—whether delivered in print, via static websites, or through periodic telehealth visits—often fail to adapt dynamically to emerging clinical evidence ( Javanparast et al., 2021 ) or to the evolving clinical status of individual patients. Moreover, existing digital health platforms seldom incorporate mechanisms to detect when their guidance may be outdated or insufficiently confident ( Wang et al., 2025 ) , leading to knowledge gaps in both patients and clinicians. Large language models (LLMs) ( Grattafiori et al., 2024 ; Abdin et al., 2024 ; Minaee et al., 2025 ) offer a potential solution by providing natural language interfaces for patients to query. However, LLMs face limitations due to fixed knowledge cutoffs ( Cheng et al., 2024 ) and the most-capable are trained on broad, general‑purpose corpora ( Alber et al., 2025 ) , leaving them unaware of the latest bariatric surgery guidelines or nuanced postoperative considerations ( Bélisle-Pipon, 2024 ) . One popular way to address outdated knowledge in LLMs is through retrieval‑augmented generation (RAG) methods ( Gao et al., 2024 ) , which retrieve up-to-date documents from a database. However, their databases are typically static and efforts to increase them over time easily introduce “context noise,” overwhelming the LLM with conflicting inputs and producing higher hallucination rates ( Zhang and Zhang, 2025 ) . Furthermore, these methods lack built‑in mechanisms to assess when their own outputs may be insufficiently confident, placing patients and clinicians at risk of incomplete or incorrect guidance ( Lewis et al., 2021 ) . We propose bRAGgen , an adaptive RAG framework that continuously monitors its response confidence and, upon detecting uncertainty, autonomously retrieves and integrates the latest peer-reviewed evidence and clinical guidelines from trusted biomedical sources such as PubMed 1 1 1 https://pubmed.ncbi.nlm.nih.gov . This self‑updating architecture ensures that guidance remains current, accurate, and clinically relevant, reducing the risk of outdated or misleading recommendations. Complementing bRAGgen , we introduce bRAGq , a curated dataset of 1,302 bariatric surgery–related questions validated by a bariatric surgeon, constituting the first large‑scale, domain‑specific benchmark for the full spectrum of MBS patient care—from preoperative preparation to long-term postoperative management. In a two-phase evaluation, leveraging both LLM‑based metrics and expert surgeon reviews, bRAGgen demonstrates substantially superior performance in generating clinically accurate, relevant, and actionable responses, paving the way for more accessible, evidence‑based support for MBS patients. Generalizable insights about machine learning in the context of healthcare While we focus on bariatric surgery patients’ educational needs, our work includes generalizable insights for other healthcare contexts. First, the need to access up-to-date medical evidence is widespread, especially in areas where the scientific literature grows quickly. Our work demonstrates that it is feasible to approach this problem by training models to directly access webpages, and keeping them relevant throughout deployment. Second, our methods are general purpose and are widely applicable. While there is a major need for bariatric surgery education and it is our expertise, our machine learning methods can be generalized to any domains where patient questions can be collected, relevant literature exists, and model answers can be validated. 2 Related Work The landscape of digital health interventions for bariatric care across the entire surgical spectrum-from preoperative preparation to long-term postoperative management has expanded considerably, with multiple studies evaluating the efficacy and usability of mobile applications and web-based platforms. For instance, a German cohort study by Wu et al. (2024c) demonstrated that mHealth follow-up via a dedicated mobile app achieved comparable outcomes to traditional in-person care across weight loss, quality of life, and nutritional status metrics. Similarly, a systematic review by Patel and Thind (2020) identified 33 usability studies of mHealth apps across surgical subspecialties, underscoring both the potential of digital tools and the persistent challenge of sustaining long-term patient engagement. In parallel, advances in large language models (LLMs) ( Grattafiori et al., 2024 ; Abdin et al., 2024 ; Minaee et al., 2025 ) have led to growing interest in their ability to distill up-to-date information ( Atri et al., 2023c ; Dey et al., 2020 ; Atri et al., 2023b ; Atri

Read the complete text