Unlocking the Potential of Past Research: Using Generative AI to Reconstruct Healthcare Simulation Models

Thomas Monks, Alison Harper, Amy Heather · 2025

Read Unlocking the Potential of Past Research: Using Generative AI to Reconstruct Healthcare Simulation Models by Thomas Monks, Alison Harper, Amy Heather (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21646.txt.

Introduction

Thomas Monks, Alison Harper, and Amy Heather wrote this health research paper, published in two thousand twenty-five, on rebuilding old simulation models with generative AI. Hospitals use discrete-event simulations to test changes in care, but the models are rarely shared, so later teams start from scratch.

The team asked an AI system to recreate two published models in free, open-source software, working only from the journal articles. Both models were rebuilt, tested, and run, complete with user interfaces. One model's published results were matched; the other's were not, most likely because the article left out key distribution details. The rebuilt models are far more complex than past AI-made ones. The authors stress that success took skilled prompting, careful testing, and expert judgment at every step. AI alone, they imply, is not yet enough.

Catalog record

CREATORThomas Monks, Alison Harper, Amy Heather
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE92 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-21646

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Unlocking the Potential of Past Research: Using Generative AI to Reconstruct Healthcare Simulation Models Thomas Monks, Alison Harper, Amy Heather Abstract Discrete-event simulation (DES) is widely used in healthcare Operations Research, but the models themselves are rarely shared. This limits their potential for reuse and long-term impact in the modelling and healthcare communities. This study explores the feasibility of using generative artificial intelligence (AI) to recreate published models using Free and Open Source Software (FOSS), based on the descriptions provided in an academic journal. Using a structured methodology, we successfully generated, tested and internally reproduced two DES models, including user interfaces. The reported results were replicated for one model, but not the other, likely due to missing information on distributions. These models are substantially more complex than AI-generated DES models published to date. Given the challenges we faced in prompt engineering, code generation, and model testing, we conclude that our iterative approach to model development, systematic comparison and testing, and the expertise of our team were necessary to the success of our recreated simulation models. Keywords: generative AI, discrete-event simulation, healthcare, model replication, model reuse 1 Introduction Operational Research (OR) has a long history of working alongside health services to use models to support decisions aiming to improve patient health outcomes and reduce service costs. Discrete-Event Simulation (DES) is the most widely applied simulation method. Given its maturity, there are now multiple reviews of DES in health ( Soorapanth et al., (2023) ; S. Liu et al., (2020) ; Vázquez-Serrano et al., (2021) ; Zhang, (2018) ; Forbus & Berleant, (2022) ) that highlight the breadth of DES application in areas that include chronic lung disease, COVID-19 planning, cancer services, neonatal pathways, emergency care, and stroke treatment services ( Harper & Mustafee, (2023) ; Gjerloev et al., (2024) ; Penn & Viana, (2024) ; Salmon et al., (2018) ; Ouda et al., (2023) ; Koca et al., (2024) ; Yakutcan et al., (2022) ; Campos et al., (2023) ) . At the time of writing, around 100 DES articles applied to health and medical problems are published in scholarly journals, conferences and books each year ( Monks et al., (2024) ) . Many of the articles reporting applied research studies directly worked with a health service to support real decisions. In each case, key outputs of the research are the conceptual and coded DES models. The conceptual model might take many forms in the report: from an unstructured narrative description to a more formal description, such as following a conceptual modelling framework ( Robinson, (2008) ) or as prescribed by a reporting guideline ( Monks et al., (2019) ; Grimm et al., (2020) ) . Whatever the approach used by the study authors, the conceptual models are published - they are available to interested parties to reuse if they choose. On the other hand, the coded model - the digital artefact that generates the results reported in the paper - is rarely made available in any form - as low as 8% ( Monks et al., (2024) ) . The coded model remains with the original authors, but over time, as authors move on, or retire, it is effectively lost to the modelling and simulation community. We note that the development of the coded model is expensive. Not only did it require modeller time to code and analyse, but it also required time from medical professionals, healthcare managers, informatics specialists, and service users (patients and their families) to provide domain expertise, support validation, and challenge assumptions or results. This cost could be voluntary or could have been supported by public funding. The publication of coded models alongside a conceptual model falls under the field of Open Science. The Facilitating Open Science in European Research (FOSTER) project 1 1 1 https://book.fosteropenscience.eu/ ( Bezjak et al., (2018) ) defines Open Science as “the practice of science in such a way that others can collaborate and contribute, where research data, lab notes and other research processes are freely available, under terms that enable reuse, redistribution and reproduction of the research and its underlying data and methods”. In this article we focus on the reuse potential of DES models of health services that can be realised through Open Science. While the availability of conceptual models supports reuse, the cost of re-coding a DES model from scratch is likely prohibitive. This is particularly true from the perspective of a health service reuser. While an academic DES study will typically run over months or years, a health service may need a model to use within weeks or potentially days. We argue that the lack of availability of coded DES models reduces the potential long-term impact of DES research in health services. Since the release of ChatGPT 2 2 2 https://chatgpt.com/ there has been excitement about the potential of Artificial Intelligence (AI) to increase productivity. Pioneering simulation research proposes a “Natural Language Processing (NLP) Shortcut” framework ( Jackson et al., (2024) ) for generating DES models. This framework combines the general coding abilities of generative AI and Large Language Models (LLMs) with prompt engineering to create a coded and reusable simulation model. To date, generative AI has only been used to code simple simulation models; for example consisting of 20-30 lines of Python code ( Jackson et al., (2024) ; Frydenlund et al., (2024) ) or to support a reflective modelling process ( Giabbanelli, (2024) ) . The level of detail and scope of these models are not reflective of DES models used in practice within health services. Such a model would, for example, require hundreds of lines of code, include more sophisticated process logic (e.g., balking, probabilistic routing, and multiple entity classes), make use of coding best practices, control pseudorandom number streams, and include a user interface. Our aim in this study is to build on the pioneering NLP Shortcut framework to investigate the potential of generative AI to recreate coded models from the conceptual model of a healthcare DES model reported in academic publications. We note that many DES studies make use of Commercial Off-the Shelf (COTS) packages for their simulation modelling. We take an Open Science approach to recreating these models and generate Python code. This code is then licensed and preserved using Open Science archives and a research compendium ( Gentleman & Lang, (2007) ; Ayllón et al., (2021) ) . 2 Aims Our study investigates the feasibility of using generative AI to recreate DES models in healthcare based on textual descriptions from the academic literature. We focus on generating models in the Python simulation package SimPy ( Team SimPy, (2024) ) , selected for its (i) compatibility with language models’ code-generating capabilities, (ii) growing adoption in health service Operational Research ( Monks & Harper, (2023) ) , and (iii) our expertise in developing SimPy models for healthcare applications ( Harper et al., (2023) ; Allen et al., (2020) ) . To assess feasibility, we engineer prompts for Perplexity.AI to generate complete Python and SimPy code that captures model logic (e.g. arrival processes, queuing, activities, sampling, and balking). Additionally, we explore generating browser-based user interfaces using Streamlit ( Streamlit, (2025) ) to enhance accessibility for non-programmers. Our research objectives are to: • Determine

Read the complete text