Regularizing Log-Linear Cost Models for Inpatient Stays by Merging ICD-10 Codes
Chi-Ken Lu, David Alonge, Nicole Richardson, Bruno Richard · 2025
Read Regularizing Log-Linear Cost Models for Inpatient Stays by Merging ICD-10 Codes by Chi-Ken Lu, David Alonge, Nicole Richardson, Bruno Richard (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03843.txt.
Introduction
Hospital cost models must stay readable, accurate, and stable year to year, but fine-grained diagnosis codes sabotage stability: thousands of ICD-10 codes appear so rarely that their fitted costs swing wildly. Chi-Ken Lu, David Alonge, Nicole Richardson, and Bruno Richard show in 2025 that the fix can be simple — merge codes by truncating them.
Cutting codes from seven characters to six or fewer shrinks the regression while keeping every diagnostic family represented. Unlike ridge penalties, nothing gets discarded. The authors prove why it works: merging predictors raises the trace of the Hessian matrix, which directly lowers the variance of the estimates. The result also explains, they note, why health systems already favor broad groupings like DRGs over raw codes in real-world risk adjustment.
Catalog record
| CREATOR | Chi-Ken Lu, David Alonge, Nicole Richardson, Bruno Richard |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-03843
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Regularizing Log-Linear Cost Models for Inpatient Stays by Merging ICD-10 Codes Chi-Ken Lu, David Alonge, Nicole Richardson, Bruno Richard Abstract Cost models in healthcare research must balance interpretability, accuracy, and parameter consistency. However, interpretable models often struggle to achieve both accuracy and consistency. Ordinary least squares (OLS) models for high-dimensional regression can be accurate but fail to produce stable regression coefficients over time when using highly granular ICD-10 diagnostic codes as predictors. This instability arises because many ICD-10 codes are infrequent in healthcare datasets. While regularization methods such as Ridge can address this issue, they risk discarding important predictors. Here, we demonstrate that reducing the granularity of ICD-10 codes is an effective regularization strategy within OLS while preserving the representation of all diagnostic code categories. By truncating ICD-10 codes from seven characters (e.g., T67.0XXA, T67.0XXD) to six (e.g., T67.0XX) or fewer, we reduce the dimensionality of the regression problem while maintaining model interpretability and consistency. Mathematically, the merging of predictors in OLS leads to increased trace of the Hessian matrix, which reduces the variance of coefficient estimation. Our findings explain why broader diagnostic groupings like DRGs and HCC codes are favored over highly granular ICD-10 codes in real-world risk adjustment and cost models. † † volume: [VOLUME # TBD] † † year: 2025 † † workshop: Machine Learning for Healthcare 1 Introduction Accurate and interpretable cost models are essential in healthcare research, as they provide critical tools for estimating, analyzing, and understanding healthcare spending patterns Layton 2017 . Among these models, linear and log-linear regression approaches are widely used due to their simplicity and high interpretability James et al. 2021 . These models typically leverage individual-level features, such as demographic and diagnostic information, prevalent in healthcare datasets, to predict outcomes like the cost of care for acute and chronic conditions. A prominent example is the Center for Medicare and Medicaid Services (CMS), which uses regression-based models in its risk adjustment methodologies for healthcare payment systems Pope et al. 2011 ; Kautter et al. 2014 . These models produce regression coefficients that form the basis for risk scores to determine reimbursements for enrollees with diverse demographics and health statuses. As in many domains, healthcare data today is high-dimensional, often involving many observations ( n ) and features ( p ). This high dimensionality can hinder a model’s ability to learn stable, robust representations—an issue commonly addressed through regularization techniques. For instance, Ridge regression introduces a penalty term that shrinks coefficients to reduce model variance, mitigating overfitting at the expense of added bias Hastie et al. 2017 . Similarly, in deep learning, pooling layers reduce feature dimensionality to facilitate efficient and meaningful representation learning Goodfellow et al. 2016 . In this context, predicting the cost of inpatient stays is especially challenging and important. Inpatient stays account for a significant portion of total healthcare expenditures in the United States Pfuntner et al. 2006 , and each stay is attached with diagnosis codes, most commonly recorded using ICD-10-CM. Modeling these costs with high accuracy and consistency is a shared concern of machine learning and healthcare communities. This study analyzes inpatient cost data from the New York Downstate subset of the Medicare Provider Analysis and Review (MedPAR) dataset. Each inpatient stay has at most 25 assigned diagnosis codes. The outcome variable y is the log-transformed cost of the stay, and the predictors {\bf x} are binary variables indicating the presence of specific ICD-10 codes. We apply ordinary least squares (OLS) regression to fit the model across randomized training subsets ( n_{\text{tr}}\approx 400{,}000 , p\approx 20{,}000 ), evaluating on a holdout test set ( n_{\text{te}}\approx 100{,}000 ). The average training R^{2} is approximately 0.45, with a test R^{2} near 0.4. Despite the predictive scores being better than those reported for HHS-HCC risk adjustment models Kautter et al. 2014 , the OLS regression coefficients are highly inconsistent across subsamples. This instability makes it difficult to use these coefficients for developing a reliable ICD-10-based risk score. The root of this inconsistency lies in the high variance of the coefficient estimates Dicker 2014 , which is governed by the diagonal entries of the inverse Hessian matrix Hastie et al. 2017 . In our setting, this Hessian matrix corresponds to the co-occurrence structure of ICD-10 codes in the data. When codes are infrequent—appearing fewer than 10 times out of 500,000 observations—the associated coefficient estimates become highly variable. We use Spearman’s rank correlation between pairs of coefficient vectors derived from different data subsamples to measure this inconsistency. We also demonstrate that by adding an identity matrix to the Hessian and thereby regularizing the solution, Ridge regression substantially improves coefficient consistency. Beyond explicit regularization, we propose a novel method of implicit regularization via ICD-10 code merging. We reduce feature dimensionality while preserving semantic structure by grouping similar diagnosis codes—effectively truncating codes to a shorter character length. We define code granularity by a maximum character length l , (CL) \leq l . Merging codes under a reduced granularity decreases the number of predictors ( p^{(l)}<p^{(l+1)} ) and results in a transformed design matrix with increased trace in the associated Hessian: \text{tr}(X^{(l+1)\prime}X^{(l+1)})\leq\text{tr}(X^{(l)\prime}X^{(l)}), (1) with equality only if the merged codes have zero co-occurrence. This trace increase is linked to improved stability in coefficient estimation. Figure 1 illustrates this merging process and highlights a case where the equality condition holds. By integrating these ideas, we present a framework that enhances the reliability of cost estimation models using ICD-10 codes, with implications for both healthcare policy modeling and robust machine learning in high-dimensional settings. Figure 1: A toy example for illustration of reducing code granularity by merging similar codes. On the left, the design matrix X^{(4)} records the diagnoses using codes with CL\leq 4 . Then the granularity is lowered by truncating codes with 4 characters to 3. The predictors for A001 and A002 are added to form the new predictor A00 in the matrix X^{(3)} on the right. The corresponding Hessian matrices are displayed in bottom. Lowering granularity (left to right) increases the trace due to the co-occurrence of merged codes in the same stay. If the marked 1 in X^{(4)} is set to 0, then the rest of marked numbers change and the traces on both sides become identical. Generalizable Insights about Machine Learning in the Context of Healthcare • A high-dimensional log-linear cost model using only binary ICD-10 code indicators as predictors can explain approximately 40% of the variance in inpatient stay costs. • The binary design matrix X yields a Hessian matrix equivalent to the occurrence and co-occurrence matrix of ICD-10 codes. • We introduce a metric to quantify the inconsistency of regression coefficients produced by unregularized linear models. • Reducing the granularity of ICD-10 codes acts as a form of implicit regularization: grouping codes by truncating their character length – without relying on clinical knowledge – improves the stability of coefficient estimates. 2 Related Work The rising cost of healthcare has driven substantial research into more accurate, robust, and equitable healthcare cost models. These models are crucial to both patients and providers: patients rely on them for financial transparency and agency, while providers depend on them for fair reimbursement. However, traditional payment systems have been shown to undercompensate providers serving vulnerable