AIM: Adaptive Intervention for Deep Multi-task Learning of Molecular Properties

Mason Minot, Gisbert Schneider · 2025

Read AIM: Adaptive Intervention for Deep Multi-task Learning of Molecular Properties by Mason Minot, Gisbert Schneider (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25955.txt.

Introduction

Drug designers often need one molecule to be good at several things at once, and those goals fight each other during training. Mason Minot and Gisbert Schneider present AIM, a training framework that learns how to referee those fights. When the learning signals from different tasks clash, a learned policy steps in to mediate, and it trains alongside the main network.

Extra guide terms in the training goal push the policy toward steady, efficient updates that favor whichever task is struggling most. On standard molecular test sets and on data for targeted protein degraders, AIM beats rival multi-task methods, most clearly when data is scarce. The learned policy also doubles as a map of how the tasks relate.

Catalog record

CREATORMason Minot, Gisbert Schneider
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-25955

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AIM: Adaptive Intervention for Deep Multi-task Learning of Molecular Properties Mason Minot, Gisbert Schneider Abstract Simultaneously optimizing multiple, frequently conflicting, molecular properties is a key bottleneck in the development of novel therapeutics. Although a promising approach, the efficacy of multi-task learning is often compromised by destructive gradient interference, especially in the data-scarce regimes common to drug discovery. To address this, we propose AIM, an optimization framework that learns a dynamic policy to mediate gradient conflicts. The policy is trained jointly with the main network using a novel augmented objective composed of dense, differentiable regularizers. This objective guides the policy to produce updates that are geometrically stable and dynamically efficient, prioritizing progress on the most challenging tasks. We demonstrate that AIM achieves statistically significant improvements over multi-task baselines on subsets of the QM9 and targeted protein degraders benchmarks, with its advantage being most pronounced in data-scarce regimes. Beyond performance, AIM’s key contribution is its interpretability; the learned policy matrix serves as a diagnostic tool for analyzing inter-task relationships. This combination of data-efficient performance and diagnostic insight highlights the potential of adaptive optimizers to accelerate scientific discovery by creating more robust and insightful models for multi-property molecular design. \workshoptitle AI for Science 1 Introduction The in silico design of molecules frequently requires the simultaneous optimization of multiple, often competing, objectives [ 1 , 2 ] . For instance, a candidate drug molecule must exhibit high binding affinity to its target while also possessing favorable pharmacokinetic properties such as aqueous solubility and metabolic stability [ 3 ] . In domains such as drug discovery, where high-quality experimental data is often scarce and costly to acquire, achieving data efficiency is crucial [ 4 , 5 , 6 , 7 ] . Multi-task learning (MTL) is a strategy to this end, as it enables a single model to learn a unified representation across related tasks [ 8 ] . However, the efficacy of MTL is often compromised by negative transfer [ 9 , 10 ] . When task objectives conflict, their gradients can pull the model in opposing directions, leading to destructive interference that degrades overall performance. As a result, MTL’s potential for data efficiency is often unrealized because the very tasks meant to help one another end up in conflict. To effectively leverage MTL, it is therefore desirable to minimize negative transfer while maximizing synergistic benefits. A considerable body of work has sought to mitigate this negative interference, laying a crucial foundation for robust multi-task optimization. Methods such as PCGrad [ 10 ] , CAGrad [ 9 ] , and Nash-MTL [ 11 ] mitigate conflicts using principled, but often static, rules. Although effective, the uniform strategy applied by these hand-crafted heuristics creates an opportunity for a learned, adaptive policy to discover more nuanced strategies for navigating these complex trade-offs. In this work, we reframe the problem from one of heuristic gradient surgery to one of learned, adaptive optimization. We propose Adaptive Intervention for deep Multi-task learning of molecular properties (AIM), an optimization framework that learns a dynamic, context-aware policy to transform conflicting gradients into a more effective update step. The policy is trained jointly with the main model by optimizing a novel augmented objective guided by a generalization signal from a held-out validation set and is augmented with dense, differentiable regularizers. Key regularizers ensure dynamic efficiency by preserving the update step’s magnitude and promoting progress on tasks with the highest current loss. We validate AIM on a 2D toy problem, the foundational QM9 dataset [ 12 ] , and a complex targeted protein degrader (TPD) ADME (absorption, distribution, metabolism, and excretion) benchmark [ 13 ] . By evaluating on data subsets to simulate real-world, data-scarce scenarios, we show that AIM’s learned policy discovers more effective optimization strategies than handcrafted heuristics, particularly in low-data regimes. 2 Related work This section lays the groundwork for our proposed method. We begin by formally introducing the MTL paradigm and its central challenge of destructive interference. We then review the foundational heuristic approaches, setting the stage for the adaptive learning-based methods we explore. Finally, we introduce the concept of a learned optimization policy as a natural extension of this prior work. 2.1 The challenge of multi-task optimization Training a single neural network for multiple scientific tasks holds the promise of improving data efficiency and discovering shared representations between physical properties. However, this MTL paradigm introduces a fundamental optimization challenge, namely navigating the trade-offs between competing task objectives. 2.2 Heuristics for mitigating gradient interference The common formulation of the MTL objective is to minimize a weighted sum of individual task losses \mathcal{L}(\theta)=\sum_{i}w_{i}\mathcal{L}_{i}(\theta) . A primary obstacle is destructive gradient interference, where gradients from different tasks \mathbf{g_{i}}=\nabla_{\theta}\mathcal{L}_{i} , pull model parameters \theta in competing directions. This interference is multifaceted: 1. Directional conflict: The literature has traditionally defined directional conflict as an instance where gradients are opposed ( \mathbf{g_{i}}\cdot\mathbf{g_{j}}<0 ) [ 10 ] . Building on this, our work proposes a more flexible definition, positing that conflict exists across a spectrum of misalignments. Even positively correlated gradients can represent a suboptimal compromise, as their vector sum may neglect the needs of other tasks. The core challenge is thus not simply to avoid opposed gradients, but to learn the optimal degree of alignment for each specific pair of tasks. 2. Magnitude conflict: A task with a large gradient norm can dominate the update, hindering progress on other tasks. Although often treated separately, magnitude and directional conflicts are intertwined; an imbalance in magnitude can exacerbate directional disputes. An effective intervention mechanism, therefore, should ideally account for both. The presence of task conflicts means that a single optimal solution is rarely feasible, leading to a Pareto front of trade-off solutions. The difficulty lies in choosing the point on this front that offers the best generalization. Whereas static heuristics address this with a fixed geometric rule, our work learns this selection process directly from the data, enabling an adaptive strategy. 2.3 Navigating the front: from heuristics to a data-driven approach A variety of methods have been developed to navigate the multi-task optimization landscape. The simplest approach, linear scalarization (LS), which minimizes the aggregate loss, often fails to converge to the Pareto front. Principled geometric and game-theoretic methods, including PCGrad [ 10 ] , CAGrad [ 9 ] , and Nash-MTL [ 11 ] , apply fixed geometric rules to gradients. Complementary to these, adaptive weighting heuristics like FAMO [ 14 ] balance tasks by adjusting their weights based on relative progress. Our work explores a more flexible paradigm in which the intervention policy itself is learned, enabling an adaptive strategy that is sensitive to the unique relationships between tasks. 3 Method: Adaptive Intervention for deep Multi-task learning (AIM) AIM reframes the resolution of gradient conflicts from a fixed heuristic to a learned, adaptive policy \Psi . This policy transforms a set of raw task gradients \{\mathbf{g}_{i}\} into a unified update vector \mathbf{g}_{\text{intervened}} . This transformed gradient is then used to update the main model parameters, \theta , in a standard optimization step. The policy’s innovation lies in its data-driven definition of gradient conflict, which is trained by optimizing a novel augmented objective. 3.1 Differentiable gradient intervention Instead of relying on a static geometric rule (e.g.,

Read the complete text