Toward Automated Regulatory Decision-Making: Trustworthy Medical Device Risk Classification with Multimodal Transformers and Self-Training
Yu Han, Aaron Ceross, Jeroen H.M. Bergmann · 2025
Read Toward Automated Regulatory Decision-Making: Trustworthy Medical Device Risk Classification with Multimodal Transformers and Self-Training by Yu Han, Aaron Ceross, Jeroen H.M. Bergmann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00422.txt.
Introduction
Yu Han, Aaron Ceross, and Jeroen Bergmann present a 2025 paper on sorting medical devices by risk. Regulators class devices by risk to guide oversight, and doing that by hand is slow. Text alone misses shape cues, while images alone miss the fine print of a device file.
The framework joins text and images in one transformer model with cross-attention between the two streams. A self-training step labels extra cases to stretch the small set of expert labels. On a real regulatory data set the model reached about ninety percent accuracy. That beat text-only and image-only rivals by wide margins. Removing either the cross-attention or the self-training step hurt scores, which backs both design choices.
Catalog record
| CREATOR | Yu Han, Aaron Ceross, Jeroen H.M. Bergmann |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-00422
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Toward Automated Regulatory Decision-Making: Trustworthy Medical Device Risk Classification with Multimodal Transformers and Self-Training Yu Han, Aaron Ceross, Jeroen H.M. Bergmann Abstract Accurate classification of medical device risk levels is essential for regulatory oversight and clinical safety. We present a Transformer-based multimodal framework that integrates textual descriptions and visual information to predict device regulatory classification. The model incorporates a cross-attention mechanism to capture intermodal dependencies and employs a self-training strategy for improved generalization under limited supervision. Experiments on a real-world regulatory dataset demonstrate that our approach achieves up to 90.4% accuracy and 97.9% AUROC, significantly outperforming text-only (77.2%) and image-only (54.8%) baselines. Compared to standard multimodal fusion, the self-training mechanism improved SVM performance by 3.3 percentage points in accuracy (from 87.1% to 90.4%) and 1.4 points in macro-F1, suggesting that pseudo-labeling can effectively enhance generalization under limited supervision. Ablation studies further confirm the complementary benefits of both cross-modal attention and self-training. We also evaluate the model’s robustness to noisy inputs and modality-specific perturbations, demonstrating resilience in low-resource settings. Beyond technical gains, we highlight applications in automated pre-screening, compliance verification, and support for harmonization frameworks such as UDI and GMDN. Our findings suggest that domain-adapted multimodal models, when designed with transparency and specificity, can serve as trustworthy decision-support tools for regulatory classification tasks. Index terms : Medical device classification; Multimodal learning; Transformer models; Regulatory science; Self-training; Trustworthy AI. 1 Introduction Medical device classification is a foundational task in regulatory compliance, inventory management, and clinical decision-making. Regulatory authorities such as the U.S. Food and Drug Administration (FDA) and China’s National Medical Products Administration (NMPA) assign risk categories (Class I, II, III) to devices based on their intended use, invasiveness, and criticality [ 1 ] . These classifications inform market approval pathways, post-market surveillance, and the stringency of clinical evidence requirements. Traditionally, classification has relied heavily on manual expert review of device descriptions and product images—a process that is labor-intensive, error-prone, and susceptible to inter-rater variability. The implications of misclassification are profound. Under-classified devices may bypass essential safety evaluations, posing risks to patients and liability challenges to manufacturers. Over-classified devices, conversely, may face unnecessary regulatory burdens, delaying access to beneficial technologies and stifling innovation. This problem affects a wide range of stakeholders, including regulatory reviewers, industry sponsors, healthcare providers, and, ultimately, patients. As the volume and complexity of medical device submissions increase—particularly with the rise of digital health products, AI-based software, and cross-border regulation—the need for scalable, robust, and interpretable classification support tools has become increasingly urgent [ 2 ] . Within the broader context of regulatory affairs, medical device classification serves as a representative use case for several reasons. First, it involves decision-making based on heterogeneous, often unstructured information sources—ranging from technical specifications and clinical indications to product images and labeling. This reflects a common pattern across regulatory tasks, where judgments must be rendered from a mix of textual, visual, and structured inputs. Second, classification interacts with other core regulatory processes, including labeling verification, predicate device matching, conformity assessment, and regulatory pathway selection. Improvements in classification, therefore, have downstream benefits across the regulatory pipeline. In recent years, multimodal machine learning has emerged as a powerful paradigm for integrating heterogeneous data sources—including text, images, audio, and video into unified representations for downstream tasks [ 3 , 4 , 5 ] . In domains such as e-commerce [ 3 ] , social media [ 4 ] , and autonomous driving [ 5 ] , joint modeling of visual and textual modalities has demonstrated significant gains in classification, retrieval, and reasoning. In healthcare, the availability of narrative descriptions and visual content (e.g., radiographs, clinical photographs) makes multimodal modeling particularly promising for diagnostic support [ 6 ] , report generation [ 7 ] , and biomedical information extraction [ 8 ] . Despite the growing success of multimodal learning in other domains, its application to regulatory classification of medical devices remains limited. Current computational approaches in regulatory affairs field are still nascent [ 9 ] . While regulatory records often contain both structured fields (e.g., registration numbers) and unstructured modalities such as textual descriptions and product images, most existing models rely solely on text-based inputs [ 10 , 11 ] . This unimodal focus overlooks complementary visual information that may be essential for accurate risk stratification. The limitation becomes especially pronounced in cases where textual descriptions are ambiguous, incomplete, or heavily reliant on domain-specific terminology that may obscure critical distinctions. To address these limitations, we propose a novel deep learning framework that integrates textual and visual data using a cross-attention Transformer architecture. Our system learns joint representations that capture fine-grained interdependencies across modalities, enabling it to reconcile semantic ambiguity and leverage complementary signals. Inspired by recent advances in vision-language modeling [ 12 , 13 ] , we also incorporate a self-training mechanism [ 14 ] to expand the labeled dataset using high-confidence pseudo-labels, thereby improving model robustness under limited supervision. Beyond methodological motivations, the integration of visual data in medical device classification is also driven by practical and industrial considerations. In real-world manufacturing and regulatory workflows, visual inspection of devices is an essential step for verifying labeling consistency, detecting counterfeit products, and ensuring compliance with international standards such as the Global Medical Device Nomenclature (GMDN) or the Unique Device Identification (UDI) system. Manufacturers frequently maintain internal product catalogs that include both text records and standardized imagery, making image-based classification a natural complement to traditional document review. For example, in large-scale factories, barcode-scanned images of devices can be automatically matched against regulatory labels to validate their assigned class. In cross-border scenarios, where documentation may be incomplete or formatted differently due to language and jurisdictional differences, visual features provide a universal modality to anchor classification decisions. Furthermore, visual models may assist in real-time inventory auditing, post-market surveillance, or customs clearance by identifying product types through photographic cues, especially in fast-moving supply chains. Importantly, as computer vision technologies continue to advance—driven by progress in transformers, contrastive learning, and foundation models—the potential for high-fidelity image understanding in medical settings will only increase. Future applications could include automated comparison of devices with prototype schematics, identification of unregistered knockoffs through visual anomaly detection, or the reconstruction of regulatory attributes (e.g., material composition, interface type) from surface-level appearance. In this context, learning joint representations from both textual and visual information not only enhances classification accuracy but also lays the groundwork for more intelligent, scalable, and globally interoperable regulatory systems. Our main contributions are four-fold: • We construct a hybrid textual feature extraction pipeline that combines TF-IDF, latent semantic analysis (LSA), and Sentence-BERT to capture both shallow and contextual semantics. • We extract domain-adapted image representations using a fine-tuned EfficientNet-B4 architecture, enhanced with attention modules for improved feature salience. • We introduce a novel cross-modal Transformer encoder that performs fusion via stacked self-attention layers, enabling the model to learn from contextual interactions between modalities. • We implement a multi-round pseudo-label self-training strategy that augments the training data with high-confidence predictions, improving generalization without additional annotations. 2 Related Work Multimodal learning has become an increasingly important paradigm for integrating heterogeneous inputs—such as text, images, and structured meta data into unified models for classification, retrieval, and reasoning tasks. This section reviews representative approaches to multimodal fusion,