Cohort-Based Active Modality Acquisition
Tillmann Rheude, Roland Eils, Benjamin Wild, Fudan University, Department of Mathematics, Computer Science, Freie Universität Berlin · 2025
Read Cohort-Based Active Modality Acquisition by Tillmann Rheude, Roland Eils, Benjamin Wild, Fudan University, Department of Mathematics, Computer Science, Freie Universität Berlin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16791.txt.
Introduction
Tillmann Rheude, Roland Eils, and Benjamin Wild introduced Cohort-based Active Modality Acquisition in twenty twenty-five. In multimodal learning, some data types for a sample may be missing, and acquiring them costs money. The question is which samples in a whole cohort deserve the extra measurement under a fixed budget — a test-time, group-level problem earlier work largely skipped.
Their strategies estimate each candidate's expected gain by imputing the missing modality, judging what it would add before paying for it. Upper-bound heuristics benchmark the choices. On data sets with up to fifteen modalities, imputation-guided picks beat entropy-based, pre-acquisition, and random selection. A demonstration on the UK Biobank cohort showed the method guiding proteomics purchases for disease prediction at scale.
Catalog record
| CREATOR | Tillmann Rheude, Roland Eils, Benjamin Wild, Fudan University, Department of Mathematics, Computer Science, Freie Universität Berlin |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 150 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-16791
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Cohort-Based Active Modality Acquisition Tillmann Rheude, Roland Eils, Benjamin Wild, Fudan University, Department of Mathematics, Computer Science, Freie Universität Berlin Abstract Real-world multimodal machine learning often faces missing, costly-to-acquire modalities, raising the problem of which samples to prioritize for additional acquisition under a budget. Prior work mainly studies per-sample or training-time acquisition while test-time, cohort-level acquisition is less explored. We propose Cohort-based Active Modality Acquisition (CAMA) , a novel test-time cohort-level modality acquisition setting, and introduce imputation-based acquisition strategies that estimate the expected utility of acquiring a missing modality, along with upper-bound heuristics for benchmarking. Experiments on datasets with up to 15 modalities demonstrate that our proposed imputation-based strategies can more effectively guide the acquisition of an additional modality for selected samples compared with methods relying solely on pre-acquisition information, entropy-based guidance, or random selection. We showcase the real-world relevance and scalability of our method by demonstrating its ability to guide the acquisition of proteomics data for disease prediction in a large prospective cohort, the UK Biobank (UKB) . Our work provides an effective approach for optimizing modality acquisition at the cohort level, enabling more effective use of resources in constrained settings. 1 1 1 Code will be published on GitHub. 1 Introduction Consider a clinical healthcare setting where all patients in a cohort undergo a standard, inexpensive set of initial examinations, such as basic blood tests and anamnesis. However, a more advanced, expensive, or invasive procedure, like genomic sequencing or specialized imaging, could offer crucial diagnostic or prognostic information for a subset of these patients [ 23 ] . Given a limited budget or capacity for the more advanced procedure, the central question becomes: which patients should receive this additional resource to maximize the overall diagnostic yield or improve treatment outcomes across the entire cohort? Figure 1 : Motivational example for CAMA determining the added value of obtaining the MRI modality. (A) A heterogeneous cohort for which each sample has P distinct modalities. (B) Instead of using the initial subset logit scores {\textnormal{s}}_{i}^{\textnormal{avail}} , a generative model f_{\textnormal{imp}} imputes the target missing modality for every patient in the cohort. This yields imputed, augmented-modality logit scores \{{\textnormal{s}}_{i,k}^{\textnormal{imp}}\}_{k=1}^{K} that approximate the logits as if that modality were available. These scores approximate {\textnormal{s}}_{i}^{\textnormal{acquired}} , i.e., the counterfactual with only the imputed modality added. (C) An AF utilizes these scores to rank samples by acquisition priority. The graph demonstrates how the global performance metric improves from the initial baseline towards the performance of a model with access to post-acquisition data, as an increasing fraction of the cohort receives the additional modality. This acquisition process is guided by the proposed strategies operating under the acquisition budget constraint \beta . For healthcare, budgets are often resource-specific rather than flexible. For example, a hospital may have a fixed capacity for one MRI scanner, or a cohort may have a specific grant for one modality. The critical decision is prioritizing access to that single resource across the cohort, and not necessarily dynamically acquiring for different modalities per patient. Concretely, for the UKB, only 10\% of the cohort do contain the proteomics modality. Notably, the proteomics modality was financed through a multi-million dollar pre-competitive consortium of 13 pharmaceutical companies, and a \pounds 20 million government top-up was later needed to finish the full cohort acquisition [ 59 , 61 , 58 ] . This highlights the remarkable need of CAMA in real-life applications , i.e., the acquisition of a single, fixed modality for multimodal learning setups. Consider a healthcare system that can afford 1,000 expensive tests for a 100,000-person cohort. The goal is to improve health outcomes across the whole population, and this typically happens through resource allocation: who receives preventive interventions, who gets enrolled in clinical trials, who is flagged for closer monitoring. These decisions depend on accurate risk stratification. A global ranking of all 100,000 individuals by predicted risk becomes the tool through which such allocation decisions are made. The question is: which 1,000 patients should we test so that our final ranking of all 100,000 is as accurate as possible? Balancing potential gains from data modalities against the costs and complexities of acquisition is not unique to healthcare. In remote sensing, for instance, decisions must be made regarding which geographical areas warrant costly high-resolution satellite imagery to supplement widely available, lower-resolution data. Likewise, in industrial quality assurance, manufacturers could decide which components from a production batch should undergo detailed, time-consuming testing in addition to rapid, standard visual inspections to effectively identify defects at a batch level. The topic of efficient data acquisition has led to several established paradigms in machine learning, such as Active Learning (AL) [ 22 ] , Active Feature Acquisition (AFA) [ 50 ] , Active Modality Acquisition (AMA) [ 29 ] , and multimodal learning with missing data [ 66 ] . However, previous research predominantly centers on optimizing acquisition for individual samples and often does not directly address test-time budget constraints for an entire cohort. Consequently, the strategic, test-time acquisition of an additional modality from a cohort perspective remains a significant, largely unaddressed gap. This setting involves deciding, for a given batch of new samples where different subsets of modalities are available, which specific samples should receive an additional, costly modality to best achieve a global objective , e.g., maximizing overall predictive performance or diagnostic accuracy for the cohort, subject to budget constraints. We hypothesize that imputation-based acquisition functions (AFs) can effectively guide resource allocation under cohort-level constraints. The main contributions of this work are as follows: • The CAMA setting We introduce and formalize CAMA , a previously unexplored setting that addresses the challenge of prioritizing which samples within a test-time cohort should undergo additional modality acquisition based on an available subset of modalities. • Development of AFs for CAMA We propose a theoretical framework, derived from established evaluation metrics , e.g., Area Under the Receiver Operating Characteristic (AUROC) and Area Under the Precision-Recall Curve (AUPRC) , that provides a foundation for developing AFs within the CAMA setting. • Architectures for CAMA We develop a novel architecture for approaching CAMA , including a) derivations of AFs by combining generative and discriminative deep learning and b) the definition of corresponding upper bounds to serve as performance benchmarks. • Comprehensive evaluation We present a comprehensive empirical evaluation of our proposed methods across several multimodal datasets, which vary in their number of modalities and application domains, with up to 100,000 samples and 15 modalities. This includes an analysis of key assumptions, upper bounds and oracle strategies, performance challenges, and robustness. 2 Related Work In the following, we contextualize our work on CAMA by reviewing the key concepts and contributions from several relevant research domains summarized briefly in Table 1 . Table 1: Comparison of acquisition paradigms. CAMA is unique due to cohort-level acquisition. Paradigm Acquisition Decision Level Time Primary Objective AL Labels Individual Training Maximize model performance AFA Features Individual Test Optimize sample-level prediction AMA Modalities Individual Test Optimize sample-level prediction CAMA Modalities Cohort Test Maximize global cohort metric Active Learning (AL) AL seeks to enhance model training by selecting unlabeled data points for annotation by an oracle [ 47 , 43 , 31 ] . Our methodology draws significantly from AL