Patient-Conditioned Adaptive Offsets for Reliable Diagnosis across Subgroups
Gelei Xu, Yuying Duan, Jun Xia, Ruining Deng, Wei Jin, Yiyu Shi, Senior Member, IEEE · 2026
Read Patient-Conditioned Adaptive Offsets for Reliable Diagnosis across Subgroups by Gelei Xu, Yuying Duan, Jun Xia, Ruining Deng, Wei Jin, Yiyu Shi, Senior Member, IEEE (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-13094.txt.
Introduction
Gelei Xu, Yuying Duan, Jun Xia, and co-authors tackle uneven skill in AI tools for medical diagnosis. A tool may work well for one group of patients and poorly for a next group. Gaps can stem from how often a disease shows up, how scans look, or how risk is spread.
Many past fixes try to hide group traits from the tool. In care work, the authors note, those traits often hold key clues for a right call. Hiding them can cut skill and trust where stakes are high. In place of hiding, the team uses patient tuned offsets that shift with each case. Their aim is calls that stay sound across groups while still using the clues each patient brings.
Catalog record
| CREATOR | Gelei Xu, Yuying Duan, Jun Xia, Ruining Deng, Wei Jin, Yiyu Shi, Senior Member, IEEE |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-13094
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Patient-Conditioned Adaptive Offsets for Reliable Diagnosis across Subgroups Gelei Xu, Yuying Duan, Jun Xia, Ruining Deng, Wei Jin, Yiyu Shi, Senior Member, IEEE Abstract AI models for medical diagnosis often exhibit uneven performance across patient populations due to heterogeneity in disease prevalence, imaging appearance, and clinical risk profiles. Existing algorithmic fairness approaches typically seek to reduce such disparities by suppressing sensitive attributes. However, in medical settings these attributes often carry essential diagnostic information, and removing them can degrade accuracy and reliability, particularly in high-stakes applications. In contrast, clinical decision making explicitly incorporates patient context when interpreting diagnostic evidence, suggesting a different design direction for subgroup-aware models. In this paper, we introduce HyperAdapt, a patient-conditioned adaptation framework that improves subgroup reliability while maintaining a shared diagnostic model. Clinically relevant attributes such as age and sex are encoded into a compact embedding and used to condition a hypernetwork-style module, which generates small residual modulation parameters for selected layers of a shared backbone. This design preserves the general medical knowledge learned by the backbone while enabling targeted adjustments that reflect patient-specific variability. To ensure efficiency and robustness, adaptations are constrained through low-rank and bottlenecked parameterizations, limiting both model complexity and computational overhead. Experiments across multiple public medical imaging benchmarks demonstrate that the proposed approach consistently improves subgroup-level performance without sacrificing overall accuracy. On the PAD-UFES-20 dataset, our method outperforms the strongest competing baseline by 4.1% in recall and 4.4% in F1 score, with larger gains observed for underrepresented patient populations. Fig. 1 : Comparison of paradigms for integrating patient attributes. (1) Standard training leads to biased performance; (2) Group-blind suppression degrades overall accuracy; (3) Our HyperAdapt enables context-aware reliability. Medical-AI systems often report high overall accuracy on curated benchmarks, yet this metric alone is insufficient to characterize real-world reliability. In practice, patient groups differ substantially in disease prevalence, imaging appearance, and underlying risk profiles, resulting in systematic variation in the visual and contextual cues required for accurate diagnosis. Standard training pipelines typically optimize a single model over aggregated data, implicitly assuming a shared underlying distribution across patient groups. As a result, population-specific patterns are diluted during training, leading to systematic performance disparities across subgroups. Models tend to perform well on majority or well-represented populations, while predictions for underrepresented groups are often less accurate or less reliable [ 35 , 6 ] . Such group-dependent errors risk exacerbating existing healthcare disparities and eroding patient trust in AI-assisted diagnosis. Consequently, a central challenge for clinical deployment is to ensure that diagnostic models achieve high average accuracy while maintaining consistently acceptable performance for each subgroup. A common response to subgroup performance disparities has been work on algorithmic fairness, which aims to reduce group-dependent errors by suppressing or removing the influence of subgroup attributes such as age and sex [ 45 , 46 ] . In application domains such as recidivism risk assessment [ 12 ] or income prediction [ 2 ] , these attributes often act as spurious correlates, and limiting their influence can effectively mitigate bias and improve statistical fairness metrics. In medical diagnosis, however, this assumption does not generally hold. Many subgroup attributes are clinically meaningful cues that shape disease presentation, progression, and risk stratification [ 4 ] . For example, in dermatology, skin type affects lesion appearance and alters the visual cues used for diagnosis [ 29 ] . Suppressing such attributes can therefore remove information essential for diagnostic reasoning, potentially degrading performance across patient groups, including those already underserved. Rather than enforcing invariance to patient attributes, an alternative perspective is to view diagnosis as a context-conditioned prediction problem, in which patient attributes inform disease priors and modulate the relevance of specific visual cues [ 34 , 33 , 49 ] . This perspective aligns with clinical practice, where clinicians routinely adjust their interpretation of imaging findings based on patient context. Existing efforts to incorporate patient attributes into diagnostic models mainly fall into two categories. One line of work introduces subgroup-specific models or branches to capture group-dependent patterns [ 43 , 33 ] . While effective at modeling in-group variations, such approaches tend to fragment medical knowledge across separate components and struggle to balance specialization with generalization in realistic clinical settings. A second line of work incorporates patient attributes through multimodal fusion, combining clinical metadata with imaging representations within a shared backbone [ 3 , 25 ] . Although fusion-based methods preserve parameter sharing, they often treat patient attributes as auxiliary features rather than context that actively shapes diagnostic reasoning, limiting their ability to modulate which visual cues are emphasized for different patients. Together, these limitations highlight the challenge of incorporating patient attributes in a way that supports context-sensitive diagnosis while preserving a unified representation of medical knowledge. To address the aforementioned challenges, we propose HyperAdapt, a framework that shifts the paradigm from group-invariance to context-conditioning (Figure 1 ), enabling patient-conditioned modulation on top of a pretrained backbone. HyperAdapt encodes patient subgroups into a compact embedding, which is then used to generate small residual offsets for selected layers of the backbone. These offsets modulate intermediate computations rather than replacing them, allowing the model to adjust its behavior based on patient context while preserving shared medical representations learned during pretraining. Directly mapping low-dimensional attribute vectors to full-dimensional parameter offsets would be parameter-intensive and prone to overfitting, particularly under limited subgroup data. To mitigate this issue, we explore several strategies to constrain model complexity, including low-rank factorization, parameter sharing across adaptation offsets, and weight sharing within the hyper-adapter. Together, these design choices enable efficient conditioning on patient attributes without fragmenting the model or substantially increasing its parameter count. Extensive experiments demonstrate the effectiveness of the proposed approach. On the PAD-UFES-20 dataset, our method improves recall by 4.1% and F1 score by 4.4% over the strongest competing baseline, with larger gains observed for underrepresented patient groups. These results show that explicitly modeling clinically meaningful heterogeneity can improve subgroup-level reliability while maintaining strong overall performance. More broadly, the proposed framework offers a scalable and efficient pathway for developing patient-conditioned medical AI systems that better align with the structure of clinical decision making. I Related Works I-A Subgroup Reliability in Medical AI: Fairness, Metadata Modeling, and Personalization Medical AI systems frequently exhibit uneven diagnostic performance across patient subgroups, raising growing concerns about reliability, safety, and equity in clinical practice. Substantial prior works have sought to understand and address subgroup-level performance disparities from different methodological perspectives. This section reviews three complementary lines of research relevant to subgroup performance in medical AI. The first line focuses on algorithmic fairness methods that aim to reduce performance gaps between groups by enforcing parity or invariance across subgroup attributes. The second line targets explicit improvement of subgroup outcomes through subgroup-wise optimization objectives or personalized modeling strategies that tailor predictions to specific patient segments. The third line treats patient attributes as auxiliary signals or metadata and incorporates them through multimodal or metadata-conditioned modeling frameworks. I-A 1 Subgroup Fairness Through Gap Reduction Fairness research traditionally frames subgroup performance as a disparity-minimization problem, aiming to optimize metrics such as Equalized Opportunity and Equalized Odds [ 18 ] . Existing approaches operate at different stages of the learning pipeline—including modifying training data distributions, constraining model representations through