Reliability-Oriented Multilingual Orthopedic Diagnosis: A Domain-Adaptive Modeling and a Conceptual Validation Framework Journal: TOIS Volume: xx xx xx 4 DOI: 10.1145/XXXXXXX CCS: Information systems Decision support systems CCS: Information systems Multilingual information systems CCS: Computing methodologies Natural language processing CCS: Computing methodologies Artificial intelligence

Danish Ali, Li Xiaojian, Sundas Iqbal, Farrukh Zaidi · 2026

Read Reliability-Oriented Multilingual Orthopedic Diagnosis: A Domain-Adaptive Modeling and a Conceptual Validation Framework Journal: TOIS Volume: xx xx xx 4 DOI: 10.1145/XXXXXXX CCS: Information systems Decision support systems CCS: Information systems Multilingual information systems CCS: Computing methodologies Natural language processing CCS: Computing methodologies Artificial intelligence by Danish Ali, Li Xiaojian, Sundas Iqbal, Farrukh Zaidi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-02266.txt.

Introduction

Danish Ali, Li Xiaojian, Sundas Iqbal, and Farrukh Zaidi test the reliability of language models for orthopedic diagnosis from free-text notes in three languages. Such models are proposed for clinical support in low-resource settings, but reliability, calibration, and safety for structured, high-risk tasks remain unclear. Notes are written in English, Hindi, and Punjabi in this study.

The authors compare multilingual transformer encoders, a fine-tuned DistilBERT baseline, and a domain-adapted model built for orthopedic text, against zero-shot instruction-tuned large models. Large models spoke fluently but showed unstable calibration and weaker reliability under structured multilingual conditions, notably in low-resource languages. The paper also gives a conceptual validation framework for judging such systems before clinical use.

Catalog record

CREATORDanish Ali, Li Xiaojian, Sundas Iqbal, Farrukh Zaidi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE101 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.02266

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Reliability-Oriented Multilingual Orthopedic Diagnosis: A Domain-Adaptive Modeling and a Conceptual Validation Framework Journal: TOIS Volume: xx xx xx 4 DOI: 10.1145/XXXXXXX CCS: Information systems Decision support systems CCS: Information systems Multilingual information systems CCS: Computing methodologies Natural language processing CCS: Computing methodologies Artificial intelligence Danish Ali, Li Xiaojian, Sundas Iqbal, Farrukh Zaidi Abstract . Large Language Models (LLMs) are increasingly proposed for clinical decision support including multilingual diagnosis in low-resource settings. However, their reliability, calibration and safety characteristics remain insufficiently understood for structured, high-risk tasks. We present a system-level analysis of multilingual orthopedic diagnosis from free-text clinical notes in English, Hindi and Punjabi. We evaluate three modeling regimes: (i) task-aligned multilingual transformer encoders, (ii) a task-fine-tuned baseline (DistilBERT), and (iii) a domain-adaptive architecture tailored to orthopedic text (IndicBERT-HPA). These models are compared with zero-shot, instruction-tuned LLMs to assess suitability for structured diagnostic classification. Results indicate that while LLMs exhibit strong linguistic fluency, they show unstable calibration and reduced reliability under structured multilingual conditions, particularly in low-resource languages. These findings are specific to zero-shot evaluation and do not imply limitations of fine-tuned models. Domain-adaptive specialization substantially improves cross-lingual discrimination and confidence behavior. IndicBERT-HPA, with language-specific orthopedic adapter heads achieves consistently strong performance across six diagnostic categories and more predictable deployment characteristics than task-only adaptation. Building on these observations, we outline a conceptual deterministic agent-based validation framework for future implementation, formalizing evidence checks, language-sensitive validation and conservative human-in-the-loop gating. Reliable multilingual clinical decision support requires specialized architecture, explicit reliability analysis, and structured validation for safety-critical systems. 1. Introduction Multilingual clinical decision support remains a persistent challenge in linguistically diverse healthcare environments ( 49 ) . In countries such as India, clinical interactions frequently occur in regional languages including Hindi and Punjabi, while electronic health records and AI-based decision support tools are predominantly English-centric ( 44 ; 20 ) . This language mismatch constrains the equitable deployment of information systems and increases the risk of misinterpretation, delayed diagnosis and inconsistent care delivery. Orthopedic diagnosis provides a structured and high-risk testbed for multilingual clinical modeling ( 2 ; 26 ) . Clinical narratives frequently contain overlapping symptoms, incomplete descriptions and heterogeneous terminology ( 16 ; 11 ) . In low-resource languages, these challenges are further amplified by limited annotated data, mixed-script input and documentation variability ( 37 ) . Aggregate performance metrics may obscure clinically meaningful failure modes including class-conditional confusion (e.g., spinal versus joint disorders) and language-conditional instability ( 21 ) . Consequently, multilingual orthopedic diagnosis should be framed not solely as a language modeling problem but as a reliability-oriented clinical prediction task ( 22 ) . Existing approaches to multilingual clinical diagnosis can be broadly categorized into three modeling regimes: (i) task-only fine-tuning of general-purpose multilingual encoders, (ii) strong supervised baselines optimized for downstream classification and (iii) domain-adaptive architectures that incorporate structured medical specialization. While multilingual models such as IndicBERT, XLM-RoBERTa ( 10 ) and mDeBERTa ( 18 ) achieve competitive results on general NLP benchmarks ( 19 ) , their behavior under safety-critical, domain-specific multilingual clinical settings remains insufficiently characterized. Furthermore, zero-shot instruction-tuned LLMs ( 50 ) , despite strong generative fluency, lack explicit calibration constraints and structured label grounding when applied to diagnostic classification tasks ( 53 ) . Despite recent progress, current paradigms exhibit several limitations in structured multilingual clinical deployment: • Insufficient domain specialization: Task-only fine-tuning does not explicitly enforce clinically discriminative subspaces, potentially limiting robustness under symptom overlap and domain-specific terminology. • Language-conditional instability: Models trained on heterogeneous corpora may demonstrate uneven performance across low-resource languages and mixed-script inputs. • Calibration gaps: Optimization via standard classification objectives does not ensure reliable confidence estimates in high-risk medical contexts. • Limited validation mechanisms: Most systems generate direct predictions without deterministic validation layers to support safe human oversight. To address these limitations, we propose IndicBERT-HPA, a domain-adaptive multilingual diagnostic framework for orthopedic classification in English, Hindi and Punjabi. IndicBERT-HPA augments IndicBERT ( 5 ) with a specialized multi-layer adapter head designed to project general linguistic representations into a clinically discriminative orthopedic subspace. We conduct a controlled comparison across three modeling regimes: (i) task-only fine-tuning of multilingual encoders, (ii) a strong supervised baseline (DistilBERT), and (iii) the proposed domain-adaptive architecture. In addition, we evaluate zero-shot large language models to examine whether generative capability translates into structured diagnostic reliability under low-resource conditions. To support deployment-oriented safety, we further introduce a deterministic agent-based validation layer that performs structured evidence checking and enables human-in-the-loop gating. Our study is grounded in a curated multilingual orthopedic dataset refined from a substantially larger raw corpus to ensure label consistency, clinical validity and ethical compliance. Through systematic experimentation and reliability-oriented evaluation, we investigate whether explicit domain specialization improves multilingual diagnostic performance and calibration compared to task-only adaptation. The main contributions of this work are as follows: (1) Curated Multilingual Orthopedic Dataset. We construct a large-scale multilingual orthopedic clinical corpus spanning English, Hindi and Punjabi. The dataset supports both class-balanced evaluation and natural clinical prevalence splits enabling controlled and deployment-oriented reliability analysis. (2) Domain-Adaptive Multilingual Architecture. We introduce IndicBERT-HPA, a task-aligned architecture that augments IndicBERT with language-specific orthopedic adapter heads. The design explicitly models domain structure while preserving cross-lingual transfer capacity. (3) Reliability-Oriented Evaluation Framework. We develop a comprehensive evaluation protocol that goes beyond aggregate accuracy, incorporating per-class analysis, calibration assessment (ECE) and cross-lingual instability examination under both balanced and real-world distributions. (4) Conceptual Design of Deterministic Framework. Motivated by empirically observed failure modes, we propose a conceptual agent-based validation framework that incorporates evidence-consistency checks, language-risk screening and conservative deferral mechanisms for safety-oriented deployment. This framework is not implemented in the present study and is intended as a future extension toward reliable clinical integration. This study aims to systematically evaluate multilingual transformer models for orthopedic diagnosis in English, Hindi, and Punjabi under controlled experimental conditions. Specifically, we investigate the distinction between task-only fine-tuning and explicit domain adaptation through the proposed IndicBERT-HPA architecture and assess how these modeling choices influence reliability in low-resource clinical settings. We further analyze the limitations of zero-shot large language models for structured diagnostic prediction, with emphasis on calibration behavior and label grounding. In addition, we examine the role of an agent-based validation layer in supporting safety-oriented deployment and human oversight. Finally, we study the impact of low-resource language constraints on multilingual clinical NLP systems including language-conditional errors and distribution-sensitive performance variation. This study addresses the following research questions: This study addresses the following research questions: • RQ1: How can multilingual clinical datasets be refined to ensure consistency, privacy preservation and diagnostic validity? • RQ2: Does explicit domain adaptation improve multilingual orthopedic diagnostic performance and calibration compared to task-only fine-tuning? • RQ3: How do fine-tuned encoders and zero-shot LLMs differ in reliability, calibration behavior and failure modes for structured diagnosis? • RQ4: Can deterministic validation mechanisms enhance safety in multilingual clinical decision support systems? The remainder of this paper is organized as follows. Section 2 reviews related work. Section 3 presents the dataset and preprocessing pipeline. Section 4 describes the proposed framework. Section 5 reports experimental results. Section 6 discusses deployment considerations and limitations. Section 7 concludes the paper. 2. Related Work This work intersects several research directions including multilingual clinical natural language processing, large language

Read the complete text