Fused Multinomial Logistic Regression Utilizing Summary-Level External Machine-learning Information

Chi-Shian Dai [email protected] of StatisticsNational Cheng Kung UniversityTainan, 701, Taiwan, Jun Shao [email protected] of StatisticsUniversity of Wisconsin-MadisonMadison, 53706, USA · 2026

Read Fused Multinomial Logistic Regression Utilizing Summary-Level External Machine-learning Information by Chi-Shian Dai [email protected] of StatisticsNational Cheng Kung UniversityTainan, 701, Taiwan, Jun Shao [email protected] of StatisticsUniversity of Wisconsin-MadisonMadison, 53706, USA (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03939.txt.

Introduction

Chi-Shian Dai and Jun Shao fused outside machine-learning guesses into multinomial logistic regression for a primary study. Outside guesses supply rich moment rules through a data-based method, robust to covariate shift under mild overlap without density-ratio modeling. The method handles rough results, partial inputs, input shift, and concept shift in outside sources. Theory gives soundness, normal shape in large samples, and strict gains under mild rules, with tests and a food-survey use.

Catalog record

CREATORChi-Shian Dai [email protected] of StatisticsNational Cheng Kung UniversityTainan, 701, Taiwan, Jun Shao [email protected] of StatisticsUniversity of Wisconsin-MadisonMadison, 53706, USA
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79198
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.03939

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Fused Multinomial Logistic Regression Utilizing Summary-Level External Machine-learning Information Chi-Shian Dai [email protected] of StatisticsNational Cheng Kung UniversityTainan, 701, Taiwan, Jun Shao [email protected] of StatisticsUniversity of Wisconsin-MadisonMadison, 53706, USA Abstract In many modern applications, a carefully designed primary study provides individual-level data for interpretable modeling, while summary-level external information is available through black-box, efficient, and nonparametric machine-learning predictions. Although summary-level external information has been studied in the data integration literature, there is limited methodology for leveraging external nonparametric machine-learning predictions to improve statistical inference in the primary study. We propose a general empirical-likelihood framework that incorporates external predictions through moment constraints. An advantage of nonparametric machine-learning prediction is that it induces a rich class of valid moment restrictions that remain robust to covariate shift under a mild overlap condition without requiring explicit density-ratio modeling. We focus on multinomial logistic regression as the primary model and address common data-quality issues in external sources, including coarsened outcomes, partially observed covariates, covariate shift, and heterogeneity in generating mechanisms known as concept shift. We establish large-sample properties of the resulting fused estimator, including consistency and asymptotic normality under regularity conditions. Moreover, we provide mild sufficient conditions under which incorporating external predictions delivers a strict efficiency gain relative to the primary-only estimator. Simulation studies and an application to the National Health and Nutrition Examination Survey on multiclass blood-pressure classification. Code is available at https://github.com/chichiihc2/MLfused.git . Chi-Shian Dai and Jun Shao † † heading: 1 2025 1- † † shortheadings: Fused Multinomial Logistic Regression / Dai and Shao † † firstpage: 1 1 1 footnotetext: Corresponding author keywords classification, concept shift, covariate shift, data fusion, empirical likelihood. 1 Introduction In recent years, researchers have increasingly moved beyond analyzing a single dataset toward integrating multiple data sources to improve statistical efficiency. In many applications, a primary study is carefully designed and provides high-quality individual-level data with a not-so-large sample size, while additional external information is available from auxiliary sources with much larger sample sizes but only summary statistics (not individual-level data). A growing body of literature has investigated how to incorporate summary-level external information ( 2 ; 13 ; 22 ; 19 ; 21 ; 23 ; 3 ; 5 ; 9 ; 11 ; 4 ; 18 ; 7 ) . In this paper, we focus on multiclass classification as the primary inferential task. We adopt multinomial logistic regression, which, in addition to prediction, provides a principled inferential framework in which regression coefficients are interpretable as log-odds ratios or contrasts for comparison. To enhance multinomial logistic regression without sacrificing its interpretability, we use extra moment conditions constructed from external summary-level predictors from modern machine-learning methods ( 12 ) such as gradient boosting, XGBoost, regression trees, random forests, and deep neural networks, which have achieved remarkable predictive success with large training datasets. These machine-learning methods are powerful and robust (nonparametric), which induces a rich class of valid moment conditions, unlike parametric methods in external sources that are often not robust against model violations. Our work mainly bridges two complementary sources of information: an interpretable multinomial logistic regression and a powerful, robust, but black-box external predictor to improve efficiency. The main challenge in leveraging external information is the heterogeneity across data sources, including covariate shift (the differences in covariate populations between the primary and external sources) and heterogeneity in outcome-generating mechanisms (the differences in conditional means of outcomes given covariates in different sources), also known as concept shift or drift ( 14 ; 8 ) . Covariate shift can be handled by estimating density ratios ( 4 ) when external individual-level data are available, or when only summary-level external information is available through some models on density ratios ( 3 ; 11 ; 18 ) or through moment selection ( 7 ) . In our approach, we handle covariate shift without estimating the density ratio, assuming that unmeasured covariates (if any) in the external source are missing at random and leveraging the fact that machine-learning methods are non-parametric, which are robust to covariate shift. The heterogeneity in outcome-generating mechanisms can arise from differences in enrollment, sampling, or case mix, which in turn can alter class proportions. To accommodate this, we divide the regression parameters into two sets: a set of free parameters representing source-specific discrepancies and another set of shared parameters allowing information transfer across different data sources. Without shared parameters, the primary and external sources are disconnected, and external information does not help. To ensure the success of transferring external information, the number of free parameters cannot exceed the effective number of moment conditions contributed by the external information. We discuss how to construct enough moment conditions after developing the methodology and related asymptotic theory. Our main contributions are fourfold. First, we propose a general data fusion/integration framework that incorporates external nonparametric machine-learning probability predictions into a primary, interpretable multiclass classification model. Second, we develop ideas to address heterogeneity in covariates and outcome-generating mechanisms across different sources. Third, the proposed framework can handle the external sources with partial covariates and coarsened labels. Lastly, we establish large-sample properties of the resulting fused estimator, including consistency and asymptotic normality under regularity conditions, and we derive mild conditions under which incorporating external predictions yields a strict efficiency gain relative to the primary-only estimator. The paper is organized as follows. Section 2 introduces the notation and formalizes the data structure arising from heterogeneous primary and external sources. Section 3 develops the proposed fused estimation methodology and establishes its key theoretical properties. Section 4 evaluates the finite-sample performance of the proposed estimator through simulation studies, demonstrating clear efficiency gains over methods that do not use external information. Section 5 illustrates the practical utility of the proposed approach using a real data example from the National Health and Nutrition Examination Survey, focusing on a multiclass blood pressure classification problem. Section 6 provides a discussion. The Appendix contains all technical proofs. 2 Data Structure We introduce the data structures for a primary study ( S=1 ) and one external study ( S=0 ). Extensions to multiple external studies are straightforward. 2.1 Primary Study Let (Y_{i},{\bm{X}}_{i}) , i=1,\ldots,n , denote n independent and identically distributed observations from (Y,{\bm{X}}) under the primary study ( S=1 ), where Y\in\{1,\ldots,K\} is a class label outcome, {\bm{X}} is a p -dimensional covariate vector whose first component is 1 (corresponding to an intercept) and the remaining components are observed features, and K\geq 2 and p\geq 2 are fixed and known. For the outcome-generating mechanism, we assume that, conditional on {\bm{X}} , the class label Y follows the multinomial logistic regression model, P(Y=k\mid{\bm{X}},S=1)=\left\{\begin{array}[]{ll}\frac{\exp(\X^\top\btheta_k)}{1+\sum_{j=1}^{K-1}\exp(\X^\top\btheta_j)}&\qquad k=1,\ldots,K-1,\\[5.69054pt] \frac{1}{1+\sum_{j=1}^{K-1}\exp(\X^\top\btheta_j)}&\qquad k=K,\end{array}\right. (1) where \bm{\theta}_{1},...,\bm{\theta}_{K-1} are source-specific regression parameters and, throughout, {\bm{x}}^{\top} denotes the transpose of vector {\bm{x}} . The target parameter to be estimated is \bm{\theta}=(\bm{\theta}_{1}^{\top},...,\bm{\theta}_{K-1}^{\top})^{\top} . 2.2 External Source Let (U_{i},{\bm{Z}}_{i}) , i=1,\ldots,N , denote N independent and identically distributed observations from (U,{\bm{Z}}) under an external study ( S=0 ). The external sample size N is much larger than the sample size n of the primary study in the sense that n/N\to 0 as n grows to \infty . In the ideal setting, (U,{\bm{Z}}) has the same form as (Y,{\bm{X}}) in the primary study, although their populations may be different. In many applications, however, the external

Read the complete text