Taming the Real-world Complexities in CPT E/M Coding with Large Language Models
Islam Nassar, Yang Lin, Yuan Jin, Rongxin Zhu, Chang Wei Tan, Zenan Zhai · 2025
Read Taming the Real-world Complexities in CPT E/M Coding with Large Language Models by Islam Nassar, Yang Lin, Yuan Jin, Rongxin Zhu, Chang Wei Tan, Zenan Zhai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-25007.txt.
Introduction
Islam Nassar leads a team with ten co-authors who use large language models to code doctor visits for billing. Such codes, from a set list, log what a doctor did at a visit. Right codes help pay flow, but the task adds to the note load on doctors and is full of real world snags.
Their frame, ProFees, was built to face those snags head on. On a set of real cases picked by experts, it beat a sold code tool by a large gap in right codes. It also beat the team's best lone prompt base by a small gap. The work lays out the snags in full, then tests the fix on each.
Catalog record
| CREATOR | Islam Nassar, Yang Lin, Yuan Jin, Rongxin Zhu, Chang Wei Tan, Zenan Zhai |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-25007
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Taming the Real-world Complexities in CPT E/M Coding with Large Language Models Islam Nassar, Yang Lin, Yuan Jin, Rongxin Zhu, Chang Wei Tan, Zenan Zhai Abstract Evaluation and Management (E/M) coding, under the Current Procedural Terminology (CPT) taxonomy, documents medical services provided to patients by physicians. Used primarily for billing purposes, it is in physicians’ best interest to provide accurate CPT E/M codes. Automating this coding task will help alleviate physicians’ documentation burden, improve billing efficiency, and ultimately enable better patient care. However, a number of real-world complexities have made E/M encoding automation a challenging task. In this paper, we elaborate some of the key complexities and present ProFees, our LLM-based framework that tackles them, followed by a systematic evaluation. On an expert-curated real-world dataset, ProFees achieves an increase in coding accuracy of more than 36% over a commercial CPT E/M coding system and almost 5% over our strongest single-prompt baseline, demonstrating its effectiveness in addressing the real-world complexities. 1 Introduction Accurate and efficient coding of Evaluation and Management (E/M) services using the Current Procedural Terminology 1 1 1 https://www.ama-assn.org/practice-management/cpt/cpt-overview-and-code-approval (CPT) system is crucial for healthcare providers, as it directly impacts billing accuracy, regulatory compliance, and revenue cycles. Traditionally, CPT E/M coding is performed by physicians and trained human coders who review clinical encounter notes and electronic health record (EHR) data to assign appropriate codes. They often rely on detailed guidelines American Medical Association 2023 to navigate the complexity of Medical Decision Making (MDM) for a given encounter. However, this manual process is resource-intensive, prone to inconsistencies due to variation in coder’s expertise, and vulnerable to errors arising from intricate coding guidelines and the nuanced nature of clinical reasoning. Many studies highlight the financial and compliance ramifications of coding inaccuracies. For example, the U.S. Office of Inspector General (OIG) estimated that in 2010, Medicare inappropriately paid US$6.7 billion (21% of total E/M payments) due to incorrect coding and/or insufficient documentation; 42% of claims were miscoded and 19% lacked proper documentation U.S. Department of Health & Human Services, Office of Inspector General 2014 . A Florida-specific analysis estimated that nearly 9% of primary care visits (2.6 million annually) were undercoded, resulting in approximately US$114 million in lost revenue for hospitals Tenpas and Dietrich 2023 . These findings underscore both widespread inconsistency and the high cost of coding errors. Automation of CPT E/M coding offers several compelling advantages, including reducing human error through consistent application of complex coding rules, achieving scalable efficiency by handling high-volume chart reviews without increasing workload on physicians and coders, enhancing timely revenue capture by improved turnaround time. Additionally, automated coding can provide transparent and traceable reasoning, crucial for audits and compliance risk mitigation and enhance coding consistency, and optimizes coder resources by allowing human coders to focus on more complex or ambiguous cases. Beyond its immediate healthcare impact, CPT E/M coding exemplifies a broader class of problems where models must map free-text to codified decisions under strict guideline constraints. Insights from this work therefore extend to other regulated domains that require auditable, rule-aligned predictions. Table 1: Key challenges in CPT E/M automation and our corresponding solutions. Challenge Description Solution Intermediate labels Production data include only the final CPT code, while the underlying medical decision-making (MDM) level with 3 intermediate element levels are absent. 117 encounters are re-annotated by our internal expert with the 3 intermediate elements – Problem, Data, and Risk levels + justifications. Label noise Even expert coders disagree: in our case, external and internal coding experts diverge on 56% of encounters (Sec. 5 ). The development data is split into Platinum (agreement) and Disagreement ; We first tune our prompts on Platinum , then iteratively refine using error analysis on both subsets with the internal expert. Explainability Clinicians and auditors require human-readable reasoning and black-box predictions are unacceptable. Chain-of-thought exemplars and checklist-based critic outputs accompany every prediction, explicitly citing CPT guideline clauses. Robustness A production system must deliver repeatable results. LLM-based method could be potential solution but outputs could be stochastic. Self-consistency ensemble ( K{=}3 ) with majority voting plus a deterministic decision tree ensures repeatable results at controlled cost. Clinical breadth Correct coding demands deep, wide-ranging familiarity with guidelines and edge cases. Dynamic few-shot retrieval injects guideline-aligned examples; iterative error analysis and prompt tuning add edge-case expertise. In this paper, we share our practical experience in developing and deploying ProFees, an automated CPT E/M coding model within a production EHR system. Development of ProFees provides solutions to five main complexities of real-world E/M coding, summarized in Table 1 . Leveraging advanced large language model (LLM) techniques, ProFees employs a dynamic few-shot prompting strategy to align coding decisions with best practices contextually relevant to each patients’ encounter notes. Furthermore, an LLM-based critic is integrated to validate reasoning across the MDM elements explicitly, ensuring robust and explainable outcomes. To mitigate the stochasticity inherent in generative models, ProFees also adopts a self-consistency strategy to achieve reliable and consistent coding predictions. This paper’s key contributions include: • Development and deployment of ProFees, a sophisticated LLM-based framework employing dynamic few-shot prompting, explicit criticism, and self-consistency techniques for accurate CPT E/M coding and intermediate MDM level estimation. • Empirical demonstration of ProFees’s efficacy on a proprietary test dataset 2 2 2 This test set was meticulously annotated by our internal expert in the same way as the development set of 117 encounters. Due to privacy and regulatory constraints, this test set is not publicly available. with comparison against a commercial CPT E/M coding system and single-prompt baselines. Our findings highlight superiority of ProFees in E/M coding accuracy over both the commerical system and our strongest baseline by around 36% and 5% respectively, validating its efficacy in CPT E/M coding automation. Figure 1: CPT E/M coding decision tree used to guide human coding of office and preventive visits. Other visit types are omitted for brevity. “SF” stands for “straightforward”. 2 Related Work CPT coding has gone through three broad phases of modeling and technical development, which are classical machine learning, deep learning and large language modeling. Classical ML models heavily rely on feature engineering, which demands significant pre-processing efforts to convert structured and/or unstructured text data into hand-crafted features. Supervised classifiers, such as support-vector machines and gradient-boosted trees, are then learned on these features to predict CPT codes Morey et al. 2025 ; Khaleghi et al. 2021 . While achieving moderate accuracy, these models struggle with the complexity and variability of clinical narratives. Deep learning models for CPT coding mainly feature encoder-decoder architectures. The encoder learns to map medical text into dense embeddings, based on which the decoder learns to predict the codes. Despite the wide success of DL models in ICD-10 coding Ji et al. 2024 , their application to CPT coding is limited and has not been reported with favorable results compared to classical ML models Burns et al. 2020 ; Levy et al. 2022 ; Kim et al. 2023b , which is likely due to the unavailability of large datasets for effective training. Recently, LLMs have broken new grounds in medical coding, where with prompt engineering, GPT-4 is reported to have achieved comparable performance to SOTA DL models for ICD-10 coding Boyle et al. 2023 . However, other