A collection of innovations in Medical AI for patient records in 2024

Yuanyun Zhang, Shi Li · 2025

Read A collection of innovations in Medical AI for patient records in 2024 by Yuanyun Zhang, Shi Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-05768.txt.

Introduction

Yuanyun Zhang and Shi Li argue for a new kind of yearly review that collects the top medical AI advances for patient records, in a 2025 paper. Healthcare AI now moves faster than journals can publish. Surveys are outdated before they print. Clinicians still need a trustworthy map of what changed. The authors propose an annualized citation framework.

Each year's paper would systematically gather that year's breakthroughs. Methods, validations, and real-world effects would be compared side by side. The 2024 collection illustrates the format in action. Large language models for records feature heavily. Hospitals gain a dated, citable snapshot instead of scattered preprints. The proposal is as much about publishing reform as about AI itself.

Catalog record

CREATORYuanyun Zhang, Shi Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-05768

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A collection of innovations in Medical AI for patient records in 2024 Yuanyun Zhang, Shi Li Abstract The field of Artificial Intelligence (AI) in healthcare is evolving at an unprecedented pace, driven by rapid advancements in machine learning and the recent breakthroughs in large language models (LLMs) ( Zhao et al. 2023 ) . While these innovations hold immense potential to transform clinical decision-making, diagnostics, and patient care, the accelerating speed of AI development has outpaced traditional academic publishing cycles. As a result, many scholarly contributions quickly become outdated, failing to capture the latest state-of-the-art methodologies and their real-world implications. This paper advocates for a new category of academic publications—an annualized citation framework—that prioritizes the most recent AI-driven healthcare innovations. By systematically referencing the breakthroughs of the year, such papers would ensure that research remains current, fostering a more adaptive and informed discourse. This approach not only enhances the relevance of AI research in healthcare but also provides a more accurate reflection of the field’s ongoing evolution. 1 Introduction Artificial Intelligence (AI) has become an integral force in shaping the future of healthcare ( Rajpurkar et al. 2022 ) , with applications spanning from predictive modeling and diagnostics ( Kasula 2021 ) to personalized medicine and automated clinical decision support. Recent advancements in deep learning, particularly the rise of large language models (LLMs) ( Zhao et al. 2023 ) , have further accelerated innovation, unlocking new possibilities for applying large scale models to biomedical research. However, this rapid progress presents a fundamental challenge: the traditional academic publishing cycle struggles to keep pace with the speed of AI development. By the time a study is peer-reviewed and published, newer, more advanced models and techniques may have already rendered its findings outdated or incomplete. This discrepancy raises critical concerns about the relevance and longevity of AI research in healthcare. While conventional papers provide valuable theoretical and empirical contributions, they often fail to reflect the field’s most current state. As AI-driven solutions continuously redefine the boundaries of what is possible, there is a growing need for an alternative publication model that prioritizes recent innovations. To address this issue, we propose a new category of academic papers that explicitly cite the breakthroughs of the year, ensuring that discussions and analyses remain grounded in the latest developments. This approach would not only improve the timeliness of AI research in healthcare but also foster a more dynamic and iterative academic ecosystem. By incorporating up-to-date references and acknowledging the fluid nature of AI advancements, such papers would enhance the accuracy, relevance, and practical impact of AI-driven healthcare research. 2 Language Models In 2024, the field of biomedical natural language processing (NLP) witnessed significant advancements with the development of several specialized language models tailored for health and biomedical applications. These models have been designed to enhance various tasks, including information retrieval, question answering, and text generation within the biomedical domain. One notable contribution is MediSwift, introduced by Thangarasa et al. 2024 . This model employs sparse pre-training techniques on domain-specific biomedical text data, achieving up to 75% weight sparsity during pre-training. This approach results in a 2-2.5x reduction in training FLOPs, leading to more computationally efficient models without compromising performance. The study demonstrates that sparse pre-training, combined with dense fine-tuning and soft prompting, offers an effective method for creating high-performing, computationally efficient models in specialized domains. In the realm of biomedical text retrieval, BMRetriever ( Xu et al. 2024b ) was developed to enhance retrieval tasks across various biomedical applications. This model series utilizes unsupervised pre-training on large biomedical corpora, followed by instruction fine-tuning on a combination of labeled datasets and synthetic pairs. Experiments across multiple biomedical tasks and datasets have verified BMRetriever’s efficacy, demonstrating strong parameter efficiency. Notably, the 410M variant outperforms baselines up to 11.7 times larger, and the 2B variant matches the performance of models with over 5B parameters. Addressing the challenge of referring ability in biomedical language models, Jiang et al. 2024 introduced a method to improve this aspect by designing a pre-training sequence that enhances the model’s capacity to refer to entities within biomedical texts. Empirical studies demonstrate that this approach improves both intra-sample and inter-sample referring abilities of auto-regressive language models in the biomedical domain, encouraging more profound consideration of task-specific pre-training sequence design for continual pre-training. Another significant development is BioMedLM ( Bolton et al. 2024 ) , a 2.7 billion parameter GPT-style autoregressive model trained exclusively on PubMed abstracts and full articles. Despite its relatively smaller size compared to models like GPT-4 ( Achiam et al. 2023 ) and Med-PaLM 2 ( Singhal et al. 2025 ) , BioMedLM demonstrates competitive performance on multiple-choice biomedical question-answering tasks. For instance, it achieves a score of 57.3% on MedMCQA (dev) ( Pal et al. 2022 ) and 69.0% on the MMLU Medical Genetics exam ( Wang et al. 2024b ) . This indicates that smaller, targeted models can serve as transparent, privacy-preserving, and economical foundations for specific NLP applications in biomedicine. Additionally, BioMistral ( Labrak et al. 2024 ) was introduced as an open-source large language model tailored for the biomedical domain. Utilizing Mistral as its foundation model and further pre-trained on PubMed Central, BioMistral has been evaluated on a benchmark comprising 10 established medical question-answering tasks in English, demonstrating its applicability in health contexts. 3 EHR Foundation Models Table 1: Summary of Foundation Models for Electronic Health Records (EHRs) in 2024 Model Key Features Applications EHRMamba ( Fallahpour et al. 2024 ) Scalable architecture using Mamba, supports long sequences, Multitask Prompted Finetuning (MTF), HL7 FHIR standard General-purpose EHR model, clinical forecasting, multi-task learning in healthcare MOTOR ( Steinberg et al. 2023 ) Time-to-event foundation model, explicit modeling of event likelihood and timing Risk stratification, chronic disease management, ICU monitoring Context Clues ( Wornow et al. 2024 ) Evaluates long-context models for clinical prediction tasks, balances computational cost with predictive accuracy Long-term patient record analysis, temporal modeling in EHRs CORE BEHRT (Optimized) ( Odgaard et al. 2024 ) Fine-tuned and optimized version of BEHRT for EHR data, rigorous evaluation Disease prediction, patient trajectory modeling CEHR-GPT ( Pang et al. 2024 ) Clinical text generation using large-scale pre-training on medical text data Automated report generation, clinical documentation, discharge summaries Event Stream GPT ( McDermott et al. 2023 ) GPT-based model capturing sequential dependencies in EHR data Disease progression modeling, treatment pathway optimization Retrieval-Enhanced Medical Prediction ( Kim et al. 2024 ) Integrates retrieval mechanisms to dynamically retrieve past patient events Context-aware clinical decision support, lifelong patient record modeling MEME ( Lee et al. 2024b ) Converts multimodal EHR data into pseudo-notes, uses embedding models for modality separation Emergency department decision support, multimodal learning for EHRs MetaGP ( Liu et al. ) 13-billion-parameter generative model integrating EHR data and medical literature Rare disease diagnosis, emergency condition management, biomedical research EHRAgent ( Shi et al. 2024 ) LLM-powered agent with code interface for autonomous data analysis Few-shot learning for EHR reasoning, multi-tabular medical problem solving In 2024, foundation models for Electronic Health Records (EHRs) have seen remarkable advancements, addressing long-standing challenges such as computational efficiency, handling long patient histories, and adapting across diverse clinical tasks ( Evans 2016 ) . As EHRs continue to serve

Read the complete text