Reproducible workflow for online AI in digital health
Susobhan Ghosh 1 *, Bhanu T. Gullapalli 1 *, Daiqi Gao 2, Asim Gazi 1, Anna Trella 1, Ziping Xu 2, Kelly Zhang 3, Susan A. Murphy 1,2 · 2025
Read Reproducible workflow for online AI in digital health by Susobhan Ghosh 1 *, Bhanu T. Gullapalli 1 *, Daiqi Gao 2, Asim Gazi 1, Anna Trella 1, Ziping Xu 2, Kelly Zhang 3, Susan A. Murphy 1,2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-13499.txt.
Introduction
Susobhan Ghosh, Bhanu Gullapalli and six co-authors propose in 2025 a reproducible workflow for online AI in digital health interventions. Online learners improve as streaming personal data arrives, but adaptability fights reproducibility. Iterative deployment defines the field.
Data across deployments must store with scientific utility. Algorithm acts must audit cleanly, and results must compare over time for trustworthy refinement. The workflow spans development, deployment, and analysis phases. Grounding comes from multiple real deployments by the team. Challenges per phase get concrete fixes. Health AI gains a shared discipline for learning systems that never stand still.
Catalog record
| CREATOR | Susobhan Ghosh 1 *, Bhanu T. Gullapalli 1 *, Daiqi Gao 2, Asim Gazi 1, Anna Trella 1, Ziping Xu 2, Kelly Zhang 3, Susan A. Murphy 1,2 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-13499
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reproducible workflow for online AI in digital health Susobhan Ghosh 1 *, Bhanu T. Gullapalli 1 *, Daiqi Gao 2, Asim Gazi 1, Anna Trella 1, Ziping Xu 2, Kelly Zhang 3, Susan A. Murphy 1,2 Abstract Online artificial intelligence (AI) algorithms are an important component of digital health interventions. These online algorithms are designed to continually learn and improve their performance as streaming data is collected on individuals. Deploying online AI presents a key challenge: balancing adaptability of online AI with reproducibility. Online AI in digital interventions is a rapidly evolving area, driven by advances in algorithms, sensors, software, and devices. Digital health intervention development and deployment is a continuous process, where implementation—including the AI decision-making algorithm—is interspersed with cycles of re-development and optimization. Each deployment informs the next, making iterative deployment a defining characteristic of this field. This iterative nature underscores the importance of reproducibility: data collected across deployments must be accurately stored to have scientific utility, algorithm behavior must be auditable, and results must be comparable over time to facilitate scientific discovery and trustworthy refinement. This paper proposes a reproducible scientific workflow for developing, deploying, and analyzing online AI decision-making algorithms in digital health interventions. Grounded in practical experience from multiple real-world deployments, this workflow addresses key challenges to reproducibility across all phases of the online AI algorithm development life-cycle. keywords xxxx, xxxx, xxxx \titlehead Research \subject Digital health, reproducibility, AI \corres Insert corresponding author name {fmtext} \rsbreak 1 Introduction Digital health interventions – or simply digital interventions – present an opportunity to address substantial health challenges. Leveraging ubiquitous technologies such as smartphones and wearable devices, these interventions are delivered directly to individuals as they go about their daily life. For example, individuals may receive reminders to take prescribed medication, prompts encouraging physical activity after long periods of inactivity, or supportive messages during periods of elevated stress. These actions—delivered through mobile notifications, SMS, or app-based interfaces—aim to promote healthy behaviors in their natural environment. Digital interventions are now used across a wide range of health contexts, including depression [ 1 , 2 ] , medication adherence [ 3 ] , substance use [ 4 , 5 , 6 , 7 ] , physical health [ 8 , 9 ] , cardiovascular disease [ 10 ] , cancer recovery [ 11 ] , and many more. Because the field of digital interventions is an evolving area, driven by rapid advances in software, sensors and wearable devices, the field is inherently dynamic, driven by a process of continuous improvement — that involves designing, testing, refining, and re-deploying interventions. Data collected during each deployment plays a dual role: (1) it supports scientific discovery by enabling the evaluation of intervention efficacy and the analysis of individual behaviors, and (2) it informs the (re)development of the intervention and any associated algorithm(s). Here, we use the term deployment to refer to the release and operation of a digital intervention in a real-world environment. This process of continuous improvement—driven by successive deployments and refinements is fundamental to how digital intervention research and practice evolve, enabling interventions to respond to not only changes in operating systems, sensors and wearable devices, but also real-world societal changes and individual diversity . Challenges in this continuous improvement process are many. Models or insights developed in one deployment may fail to transfer effectively to subsequent ones, due to changes in population characteristics, social dynamics, technological platforms, intervention delivery mechanisms and environmental contexts. Second, even within a single deployment, individuals differ widely in their needs, preferences and circumstances affecting their engagement with and response to interventions. To address these challenges, digital interventions have begun utilizing online artificial intelligence (AI) decision-making algorithms [ 12 , 13 , 14 ] . Here, the term online refer to AI decision-making algorithms that leverage streaming within and between individual data—such as passive sensing and self-reports— to learn parameters in a model for health outcomes and update estimates of these parameters repeatedly as new data becomes available during deployment, in contrast to offline algorithms, that learn model parameters on historical data which then remain fixed throughout deployment. Online AI decision making algorithms make use of this online learning to make decisions regarding which and when to deliver interventions during deployment. Figure 1: Adaptive feedback loop of an online AI decision-making algorithm. Data collected during deployment informs the algorithm’s intervention decisions—such as sending a prompt, a supportive message, or a behavioral recommendation—which in turn influence the individual’s future behavior and the data subsequently collected. Unlike traditional data collection settings, digital interventions employing online AI decision-making algorithms operate within an adaptive feedback loop: past data informs the algorithm’s current intervention decisions—such as sending a prompt or a message—which in turn influence individual biobehavioral outcomes and thus the data collected in the future (see Fig 1 ) [ 15 ] . This dynamic interdependence introduces tightly coupled system behaviors, where seemingly minor changes can cascade into substantial shifts in data distributions, individual engagement, and intervention outcomes. As a result, the need for reproducibility becomes especially acute in such systems. The continuous improvement of digital intervention deployments relies on the ability to learn from past deployments – using collected data and observed outcomes to refine interventions and algorithms. For this process to support cumulative scientific progress and continuous data-driven improvements, the intervention system must produce data that is reproducible . This paper follows the National Academies of Science, Engineering, and Medicine Consensus Report definition of reproducibility as the ability to obtain consistent results using the same input data, computational steps, methods, and conditions of analysis [ 16 ] . Critically, reproducibility is a prerequisite for replicability , which is the ability to achieve consistent results across deployments aimed at answering the same scientific question but using independent data and potentially in a different implementation setting. While the National Academies definition provides a broad foundation, reproducibility in online AI digital interventions is multifaceted. This paper distinguishes three complementary forms of reproducibility in online AI for digital health: design reproducibility , deployment reproducibility , and inference reproducibility . • Design reproducibility : ability to rerun algorithm development and evaluation under identical conditions—for instance, replicating results in a digital twin testbed given the same input data, computational steps, and random seeds. It can be measured by verifying whether identical simulations and training pipelines yield identical outputs. • Deployment reproducibility : ability to reproduce system behavior during real-world operation—for instance, ensuring that identical inputs and model states yield identical decisions at runtime. It can be measured by verifying that decisions are fully traceable and auditable through logs, version control, and monitoring systems, including replaying logged states to confirm identical outputs or stress-testing monitoring systems to ensure that decision failures are consistently captured • Inference reproducibility : ability to obtain consistent scientific findings from deployment data, including analyses of algorithm performance and behavioral outcomes. These terms are introduced here for completeness, but the remainder of this paper focuses primarily on design and deployment reproducibility, as inference reproducibility with regards to inference based on analyses of existing data has been addressed in some detail in the literature, for examples, see [ 17 , 18 ] . Reproducibility in the setting of online AI in digital