MAdam: Metric-Aware Multi-Objective Adam

Fengbei Liu, Rachit Saluja, Sunwoo Kwak, Ruibo Wang, Ruining Deng, Heejong Kim, Johannes C. Paetzold, Mert R. Sabuncu, [0.5em] Cornell Tech · 2026

Read MAdam: Metric-Aware Multi-Objective Adam by Fengbei Liu, Rachit Saluja, Sunwoo Kwak, Ruibo Wang, Ruining Deng, Heejong Kim, Johannes C. Paetzold, Mert R. Sabuncu, [0.5em] Cornell Tech (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03904.txt.

Introduction

Liu and co-authors fix a mismatch between multi-objective solvers and the Adam optimizer in MAdam. Multi-objective methods balance losses, gradients, or Pareto trade-offs. Nearly all hand their merged direction to Adam. Two systematic gaps open at that handoff. Adam's second-moment term tangles the time-varying preference with gradient history. Distinct trade-offs collapse toward a uniform mix.

Adam's adaptive metric also distorts the geometry solvers assume. Aligned objectives can look like conflicts after the distortion. MAdam wraps the handoff without changing solver or optimizer. It preconditions the merged direction by preference-conditioned curvature. On that whitened input, Adam's second moment collapses to identity. The realized update follows the intended metric. Across multi-task learning, Pareto recovery, physics-informed networks, and medical imaging, MAdam improves on Adam for every solver family.

Catalog record

CREATORFengbei Liu, Rachit Saluja, Sunwoo Kwak, Ruibo Wang, Ruining Deng, Heejong Kim, Johannes C. Paetzold, Mert R. Sabuncu, [0.5em] Cornell Tech
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.03904

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MAdam: Metric-Aware Multi-Objective Adam Fengbei Liu, Rachit Saluja, Sunwoo Kwak, Ruibo Wang, Ruining Deng, Heejong Kim, Johannes C. Paetzold, Mert R. Sabuncu, [0.5em] Cornell Tech Abstract Multi-objective optimization (MOO) underlies many machine learning problems, yet MOO solvers across the loss-balancing, gradient-balancing, and Pareto-based families almost universally hand their reconciled directions to Adam [ 20 ] . We show this coupling introduces two systematic gaps between the solver’s intent and the optimizer’s execution. The first is a weighting mismatch : Adam’s second-moment denominator entangles the time-varying preference vector with gradient statistics, marginalizing the preference into a history average and collapsing distinct Pareto trade-offs toward a near-uniform mixture. The second is a geometric mismatch : Adam’s adaptive metric distorts the Euclidean geometry MOO solvers assume, turning aligned objectives into apparent conflicts. To resolve both jointly, we introduce MAdam (Metric-Aware Multi-Objective Adam), a drop-in wrapper that leaves both solver and optimizer unchanged. MAdam preconditions the reconciled direction by the preference-conditioned curvature of the scalarized objective; on this whitened input, Adam’s second moment collapses to identity, so the realized update is governed by the preference-conditioned metric. Across multi-task learning, Pareto-front recovery, physics-informed neural networks, and medical imaging, MAdam consistently improves over Adam for every solver family. Code is at link . 1 Introduction Many practical learning problems require a shared model to satisfy multiple objectives simultaneously. Such settings are naturally cast as multi-objective optimization (MOO). Three MOO solver paradigms have emerged [ 4 ] : loss-balancing methods adapt objective weights to rebalance gradient magnitudes [ 44 , 39 , 30 , 28 ] ; gradient-balancing methods modify objective gradients to reduce inter-objective conflict [ 58 , 29 , 32 ] ; and Pareto-based methods steer the update toward a Pareto front target region [ 27 , 43 , 9 , 3 ] . Most gradient based MOO solvers aim to produce a single reconciled direction. However, gradient-based optimization relies on two steps: constructing the descent direction and executing the update with a suitable metric, which is a role all three MOO solver families above unanimously delegate to Adam [ 20 ] . Adam implicitly imposes its own adaptive diagonal metric via the second-moment denominator, which under stationarity approximates a diagonal empirical Fisher information matrix [ 21 , 37 ] . We show this default pairing introduces two systematic mismatches between what the solver intends and what the optimizer actually executes. Firstly, in loss-balancing and Pareto-based methods, the solver’s reconciled direction depends on a time-varying preference vector \bm{\lambda}^{(t)} : loss-balancing methods adapt these preferences to rebalance gradient magnitudes, while Pareto-based methods adjust them to steer toward a target Pareto region. However, Adam’s running second moment entangles these preferences with the objective gradient statistics into a single surrogate, weakening the intended preference strategy and collapsing distinct Pareto trade-offs into a near-uniform average (Proposition 1 ). Secondly, MOO solvers typically assume a raw Euclidean geometry: loss-balancing methods explicitly weight the objectives through linear scalarization. Gradient-balancing methods measure gradient similarities and project to reduce conflict in Euclidean space. Pareto-based methods steer a Pareto target region through linear scalarization of user preferences. However, Adam is not metric-neutral: its second-moment denominator imposes a diagonal RMS metric on every update, conflicting with the Euclidean metric assumed by MOO solvers. This geometric mismatch distorts the solver’s reconciled direction, turning aligned objectives into apparent conflicts and vice versa (Proposition 2 ). These observations raise two questions: Q1. Which metric should govern MOO updates? Q2. How can it be realized within the existing solver–Adam pipeline? We propose MAdam (Metric-aware Multi-objective Adam), which addresses both questions with a single mechanism. For Q1, we derive the preference-conditioned diagonal Fisher information matrix of the scalarized objective at the active preference, given by the second moment of the scalarized gradient and admitting a decomposition into within- and cross-objective Fisher blocks. For Q2, we precondition the solver’s reconciled direction by this diagonal metric prior to the Adam update, modifying neither the solver nor the optimizer. Applying the correction at Adam’s input keeps its first- and second-moment EMAs mutually consistent, so the realized parameter update is governed by the preference-conditioned metric. Our contributions are threefold • Diagnosis of the solver–Adam mismatch. We prove two failure modes of solver–Adam coupling in MOO: a weighting mismatch that marginalizes the preference (Proposition 1 ) and a geometric mismatch that distorts the Euclidean geometry assumed by MOO solvers (Proposition 2 ). • Metric-aware gradient preconditioning (MAdam). We derive the preference-conditioned diagonal Fisher of the scalarized objective and apply it as a plug-and-play preconditioner on the reconciled direction, resolving both mismatches with a single mechanism. • Empirical validation. Across diverse MOO solvers, MAdam consistently improves over Adam on multi-task learning, Pareto-front recovery, PINN benchmarks, and medical imaging applications. 2 Related Work Multi-Task and Pareto MTL. Multi-task learning reconciles competing objectives by rescaling per-objective losses, manipulating task gradients, or targeting a preference-conditioned point on the Pareto front. Representative methods range from fixed or learned scalarizations (linear scalarization, uncertainty weighting [ 19 ] ) through dynamically adapted weights (DWA [ 33 ] , IMTL [ 32 ] , CAGrad [ 29 ] ) to preference-conditioned models that recover the full Pareto front from a single network (PaMaL [ 8 ] , PaLoRA [ 9 ] ). Across this family the constructed direction is ultimately consumed by Adam, whose diagonal second-moment EMA entangles the time-varying preference \bm{\lambda}^{(t)} with per-objective gradient statistics and imposes a task-agnostic RMS metric on each step. PINN and MIA Loss Balancing. Physics-informed neural networks (PINNs) and medical-image analysis (MIA) exhibit a closely related imbalance: residual, boundary, and reconstruction losses span widely different scales, so a single objective dominates absent explicit rebalancing. The PINNacle benchmark [ 14 ] consolidates remedies along three axes, loss reweighting (LRA [ 51 ] , NTK [ 52 ] ), residual-based sampling (RAR [ 54 ] ), and adaptive activations (LAAF/GAAF [ 17 ] ); in MIA, vanilla Adam [ 40 ] and random loss weighting [ 25 ] remain the prevailing choices. Such first-order remedies adjust the descent direction yet leave the optimizer’s step metric, supplied by Adam, untouched. Curvature-Aware Optimizers. A separate line of curvature-aware optimizers, spanning natural gradient [ 1 , 37 ] , K-FAC [ 38 ] , Shampoo [ 13 ] , AdaHessian [ 57 ] , Sophia [ 31 ] , and FAdam [ 16 ] , targets a single scalar objective and supplants the first-order optimizer wholesale. None preconditions the reconciled direction of a multi-objective solver under a time-varying preference. MAdam occupies this gap: it derives a preference-conditioned diagonal Fisher of the scalarized objective and inserts it into the solver-then-Adam pipeline as a wrapper, leaving both the MOO solver and the optimizer intact. A comprehensive listing of MTL, PINN, MIA, and curvature-aware methods is in App. A . 3 Method Sec. 3.1 defines preliminaries about MOO solver and Adam. Sec. 3.2 identifies two problems Adam has on reconciled MOO directions: a weight mismatch and a geometric mismatch. Sec. 3.3 derives the correct preference-conditioned curvature (Q1). Sec. 3.4 applies it by preconditioning the solver’s output before passing it to Adam (Q2). Sec. 3.5 describes the practical implementation, and Algorithm 1 states the full procedure. 3.1 Preliminaries MOO Solver Setup. We consider C objectives with

Read the complete text