A Discordance-Aware Multimodal Framework with Multi-Agent Clinical Reasoning
Pegah Ahadian, Mingrui Yang, Sixu Chen, Xiaojuan Li, Qiang Guan · 2026
Read A Discordance-Aware Multimodal Framework with Multi-Agent Clinical Reasoning by Pegah Ahadian, Mingrui Yang, Sixu Chen, Xiaojuan Li, Qiang Guan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16333.txt.
Introduction
Ahadian, Yang, Chen, Li, and Guan present a discordance-aware multimodal design for knee osteoarthritis, where imaging damage and patient-told pain often disagree. That mismatch muddles clinical reading and patient grouping, and older choice support models it poorly. Their design joins machine learning guessing with tool-grounded multi-agent reasoning, trained on FNIH group starting data. It predicts joint space loss only and pain only progression, fusing tabular, MRI, and X-ray expert models through a stacking ensemble.
Catalog record
| CREATOR | Pegah Ahadian, Mingrui Yang, Sixu Chen, Xiaojuan Li, Qiang Guan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50798 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.16333
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Discordance-Aware Multimodal Framework with Multi-Agent Clinical Reasoning Pegah Ahadian, Mingrui Yang, Sixu Chen, Xiaojuan Li, Qiang Guan Abstract Background: Knee osteoarthritis (OA) frequently exhibits discordance between structural damage observed in imaging and patient-reported symptoms such as pain. This mismatch complicates clinical interpretation and patient stratification and remains insufficiently modeled in existing decision-support systems. Methods: We propose a discordance-aware multimodal framework that combines machine learning prediction models with a tool-grounded multi-agent reasoning system. Using baseline data from the FNIH Osteoarthritis Biomarkers Consortium (600 knees), we trained multimodal models to predict two progression tasks: (i) joint-space-loss-only progression versus non-progression and (ii) pain-only progression versus non-progression. The predictive system integrates three modality-specific experts: a CatBoost tabular model using demographic, radiographic, MRI-derived scalar, and biomarker features; MRI image embeddings extracted using a ResNet18 backbone; and X-ray embeddings derived from the same architecture. Expert predictions are fused using a stacking ensemble. Residual-based models estimate expected pain from structural features, enabling the computation of a pain–structure discordance score between observed and expected symptoms. A multi-agent reasoning layer interprets these signals to assign clinically interpretable OA phenotypes and generate phenotype-specific management recommendations. Results: Using 5-fold stratified cross-validation with out-of-fold evaluation, the full multimodal stacking model combining tabular variables, MRI and X-ray embeddings, and biochemical biomarkers achieved the best performance. For the JSL-only progression task, the model achieved AUC 0.702. For pain-only progression, the model achieved AUC 0.611. Imaging embeddings alone provided limited predictive signal, whereas clinically interpretable radiographic and MRI scalar features contributed stronger discrimination. Multimodal fusion improved performance by integrating complementary structural and biochemical information. Conclusions: Multimodal fusion of tabular clinical variables, imaging-derived features, deep image embeddings, and biochemical biomarkers improves structural progression prediction in knee OA. By coupling this prediction layer with explicit pain–structure discordance modeling and a tool-grounded multi-agent reasoning framework, the proposed architecture supports interpretable phenotype assignment and structured clinical decision support for osteoarthritis management. K eywords knee osteoarthritis, multimodal learning, discordance analysis, multi-agent systems, CatBoost, clinical decision support 1 Introduction Knee osteoarthritis (OA) is a heterogeneous chronic musculoskeletal disease in which structural degeneration, pain, and functional impairment often evolve along partially independent trajectories [ 12 , 17 , 18 ] . In routine care, clinicians are frequently confronted with patients who report substantial pain despite limited radiographic damage, as well as patients with marked structural progression but comparatively mild symptoms. This discordance complicates diagnosis, prognostic stratification, and treatment planning because conventional clinical reasoning often assumes a tighter alignment between imaging severity and symptom burden than is observed in practice [ 1 , 5 , 6 ] . Recent advances in multimodal machine learning have created new opportunities to integrate heterogeneous OA data sources [ 8 , 9 , 10 , 15 ] , including demographic covariates, radiographic features, MRI-derived quantitative measures, biochemical biomarkers [ 7 ] , and deep image embeddings. At the same time, most existing OA prediction pipelines remain optimized for a single endpoint and typically collapse structural and symptomatic information into one predictive target. Such formulations may improve benchmark metrics, but they obscure an important clinical question: whether the patient’s symptoms are concordant with, or disproportionate to, the level of structural disease. In OA, that distinction is clinically meaningful because it may indicate pain sensitization, early structural disease, compensatory adaptation, or other non-structural drivers of disability. In parallel, multi-agent large language model systems have emerged as a promising design pattern for complex clinical reasoning [ 4 ] . AutoGen formalizes this idea as a framework of conversable and customizable agents with unified send/receive interfaces, auto-reply mechanisms, and flexible conversation programming, including both static and dynamic multi-agent interaction patterns [ 2 ] . These properties make multi-agent systems appealing for tasks in which different evidence streams must be interpreted separately and then reconciled through explicit dialogue rather than collapsed into a single monolithic response [ 11 , 13 , 14 , 15 ] . Within knee osteoarthritis specifically, KOM demonstrated that a multi-agent architecture can support broader KOA management workflows through an Assessment Agent, a Risk Agent, and a Therapy Agents Group, and that clinician-AI collaboration can improve workflow efficiency and plan quality in simulated clinical settings [ 3 ] . However, KOM is designed for end-to-end KOA evaluation and treatment planning. It does not make pain-structure discordance the primary representation to be detected, debated, and interpreted. This study addresses that gap. We propose a discordance-aware multimodal framework in which structural progression prediction and symptom expectation modeling are explicitly separated. First, we develop multimodal predictive models for divided progression tasks using tabular features, radiographic and MRI-derived scalar measurements, deep MRI and X-ray embeddings, and biochemical biomarkers. Second, we estimate expected symptom burden from structural information and define discordance scores as residuals between observed and structure-expected pain or function. Third, we introduce a tool-grounded multi-agent reasoning layer in which specialized agents discuss whether structural risk and symptom burden are concordant or discordant, and then assign phenotype-level interpretations and management recommendations. The central hypothesis is that OA decision support should not merely predict progression, but should explicitly reason about disagreement between structural and symptomatic signals. Under this view, discordance is not residual noise. It is a clinically informative signal that can drive phenotype assignment, management prioritization, and auditably structured discussion among specialist agents. Our contributions are fourfold. First, we present a multimodal progression prediction framework for divided OA progression tasks using a mixture-of-experts design. Second, we formalize pain-structure discordance using residual-based symptom expectation models. Third, we design a multi-agent conversation protocol in which structuralist, physiologist, and lead consultant agents synthesize concordant and discordant evidence. Fourth, we provide a clinician-grounded evaluation setup comparing deterministic, single-agent, and decomposed multi-agent reporting configurations, with specialist-output specifications and blinded clinician review packets prepared for supplementary analysis. 2 Methods 2.1 Data sources and cohort definition The source Clinical_FNIH table contains 600 knees [ 7 ] . After exclusion of knees with missing predictors required for the progression modeling pipeline, 545 knees remained in the structural modeling cohort. A smaller complete-case subset of 90 knees with all variables required for residual-based discordance estimation was used for symptom expectation modeling and threshold analysis, table 1 . Each record corresponds to a knee-level observation. Baseline predictors include demographic variables, radiographic structural indicators, MRI-derived scalar features, and biochemical markers. In addition, MRI and X-ray images were processed to derive deep image embeddings used in multimodal fusion experiments. Table 1: Baseline demographic and clinical characteristics of the FNIH OA Biomarkers (Clinical_FNIH dataset [ 7 ] ). Characteristic Category / value Frequency (%) Knees (records) Total knees in Clinical_FNIH 600 (100.0) Side Right knee 322 (53.7) Left knee 278 (46.3) CASE / GROUPTYPE JSL and pain progressor (Case = 1) 194 (32.3) JSL only progressor (Case = 2) 103 (17.2) Pain only progressor (Case = 3) 103 (17.2) Non–progressor (Case = 4) 200 (33.3) Baseline Kellgren–Lawrence grade (V00XRKL; 0–4) Grade 1 75 (12.5) Grade 2 306 (51.0) Grade 3 219 (36.5) Baseline medial JSN (V00XRJSM; OARSI 0–3) Grade 0 162 (27.0) Grade 1 219 (36.5) Grade 2 219 (36.5) Baseline lateral JSN (V00XRJSL; OARSI 0–3) Grade 0 587 (97.8) Grade 1 13 (2.2) Race by sex 1 1 1 Race counts are reported exactly as in the Clinical_FNIH descriptive documentation; percentages in this