Multi-Objective Bilevel Learning

Zhiyao Zhang, Zhuqing Liu, Xin Zhang, Wen-Yen Chen, Jiyan Yang, Jia Liu · 2025

Read Multi-Objective Bilevel Learning by Zhiyao Zhang, Zhuqing Liu, Xin Zhang, Wen-Yen Chen, Jiyan Yang, Jia Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-07824.txt.

Introduction

Zhiyao Zhang, Zhuqing Liu, and Xin Zhang formalize learning problems with two layers of conflicting goals. Modern systems stack decisions: an upper level chooses preferences or designs, a lower level optimizes within them, and several objectives pull against each other at once. Multi-objective bilevel learning names that structure, but its theory is young.

The study builds theoretical and algorithmic foundations, clarifying when such problems are tractable and how preference guidance at the upper level shapes lower solutions. Many applications, from hyperparameter tuning to policy design, hide this shape. Naming and analyzing it gives practitioners algorithms instead of improvisation when layered goals collide.

Catalog record

CREATORZhiyao Zhang, Zhuqing Liu, Xin Zhang, Wen-Yen Chen, Jiyan Yang, Jia Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE110 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-07824

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Objective Bilevel Learning Zhiyao Zhang, Zhuqing Liu, Xin Zhang, Wen-Yen Chen, Jiyan Yang, Jia Liu Abstract As machine learning (ML) applications grow increasingly complex in recent years, modern ML frameworks often need to address multiple potentially conflicting objectives with coupled decision variables across different layers. This creates a compelling need for multi-objective bilevel learning (MOBL). So far, however, the field of MOBL remains in its infancy and many important problems remain under-explored. This motivates us to fill this gap and systematically investigate the theoretical and algorithmic foundation of MOBL. Specifically, we consider MOBL problems with multiple conflicting objectives guided by preferences at the upper-level subproblem, where part of the inputs depend on the optimal solution of the lower-level subproblem. Our goal is to develop efficient MOBL optimization algorithms to (1) identify a preference-guided Pareto-stationary solution with low oracle complexity; and (2) enable systematic Pareto front exploration. To this end, we propose a unifying algorithmic framework called w eighted- C hebyshev m ulti- h yper- g radient- d escent ( 𝖶𝖢 \mathsf{WC} - 𝖬𝖧𝖦𝖣 \mathsf{MHGD} ) for both deterministic and stochastic settings with finite-time Pareto-stationarity convergence rate guarantees, which not only implies low oracle complexity but also induces systematic Pareto front exploration. We further conduct extensive experiments to confirm our theoretical results. 1 Introduction 1) Background and motivation: As machine learning (ML) applications grow increasingly complex, modern ML training frameworks often need to address multiple objectives that are potentially conflicting. As an example, in the design of online shopping recommender systems, the objective of a training loss function could aim to prioritize lower prices, more popular brand names, or faster delivery speeds, all of which may be in conflict with each other. Moreover, such objective functions could further depend on an optimal solution from a separate and coupled optimization task, thus making the overall training task bilevel in nature. For instance, the multi-objective trainings in the aforementioned online shopping recommender system could all rely on a joint pre-training of a set of shared model parameters that define all objectives. Similar multi-objective bilevel problem structures also arise from multi-agent reinforcement learning with an actor-critic framework ( Ji, Yang, and Liang 2021 ; Yang, Ji, and Liang 2021 ; Zhang et al. 2024 ; Qiu et al. 2023 ) and the pretraining-finetuning pipeline of multi-task adaptation of a foundation model ( Chakraborty et al. 2024 ; Shen, Yang, and Chen 2024 ) . These complex problem structures create a compelling need for multi-objective bilevel learning (MOBL). Mathematically, an MOBL problem can be formulated in the following general form: \displaystyle\min_{x\in\mathbb{R}^{p}}\Phi(x):=[\phi_{1}(x),\dotsc,\phi_{S}(x)] (1) \displaystyle\text{s.t. }y^{*}(x)\in\argmin_{y\in\mathbb{R}^{q}}g(x,y), where \phi_{s}(x):=f^{(s)}(x,y^{*}(x)) , s\in[S] , represents an upper-level (UL) objective function with an implicit UL decision variable y^{*}(x) being an optimizer of a lower-level (LL) subproblem. Clearly, MOBL combines the problem structures of multi-objective optimization (MOO) and bilevel optimization (BLO), both of which are challenging ML optimization paradigms in their own right and have attracted a significant amount of attention in the research community in recent years. Although numerous algorithms have been proposed for solving MOO and BLO problems (e.g., ( Konak, Coit, and Smith 2006 ; Sener and Koltun 2018 ; Yang et al. 2024a ) for MOO and ( Ji, Yang, and Liang 2021 ; Yang, Ji, and Liang 2021 ; Zhang et al. 2024 ; Qiu et al. 2023 ) for BLO, respectively; see Section 2 for in-depth discussions), the field of MOBL remains in its infancy, with many important open problems yet to be addressed. This gap between current MOBL research state and practical needs of MOBL arises primarily from the following technical challenges. 2) Technical challenges: Solving MOBL problems is, surprisingly, significantly more challenging than a straightforward combination of applying existing MOO and BLO solution techniques, which is due to the following two key factors: First , being both multi-objective and bilevel, an MOBL problem not only inherits all the challenges of MOO and BLO, but also sees many unique and new complications unseen in either MOO or BLO. Specifically, due to the finite-time constraint in solving the LL problem under the bilevel structure of MOBL problems, one typically has an inexact y_{k} at each iteration instead of the true solution y^{*}(x_{k}) . This inaccuracy in y_{k} subsequently results in a biased estimate \hat{\nabla}\phi_{s}(x_{k}) , which could deviate from a common descent direction computed according to \nabla\phi_{s}(x_{k}) for all objectives. Beyond the cascaded error in \nabla\phi_{s}(x_{k}) , the inaccuracy can accumulate over iterations, causing the solution trajectory to drift significantly from the desired target. Second , preference-guided Pareto front (i.e., the set of all Pareto-optimal solutions) exploration in MOBL is even more challenging (see solution philosophies in Section 3 for more details), since the Pareto stationarity (necessary condition of Pareto-optimality) and preference alignment should be balanced in the algorithmic design, which is highly susceptible to the accumulative cascaded errors inherent in the bilevel structure. Moreover, although existing works have incorporated scalarization methods to handle preference alignment, conventional rescaling strategies no longer apply due to the bilevel nature of MOBL problems, thereby necessitating novel technical designs. 3) Main contributions: In this paper, we propose a series of new MOBL algorithms to overcome the aforementioned challenges, all of which enjoy a fast finite-time convergence rate guarantees and hence low oracle complexities. Collectively, our results contribute toward establishing a theoretical foundation for MOBL. Our main contributions and key results are summarized as follows: • We propose a unifying MOBL algorithmic framework called w eighted- C hebyshev m ulti- h yper- g radient- d escent ( \mathsf{WC} - \mathsf{MHGD} ) for both deterministic and stochastic settings. Our \mathsf{WC} - \mathsf{MHGD} framework offers provable fast finite-time convergence rates and low oracle complexities of \mathcal{O}(\epsilon^{-1}) and \mathcal{O}(\epsilon^{-2}) for achieving an \epsilon -Pareto stationary solution in deterministic and stochastic settings, respectively. To our knowledge, our finite-time convergence and oracle complexity results are the first of their kind in the MOBL literature involving preference. • It is worth pointing out that the algorithmic design and theoretical convergence analysis of our proposed \mathsf{WC} - \mathsf{MHGD} framework are highly non-trivial and far from straightforward extensions of traditional MOO and BLO techniques. Notably, we rigorously show that our new dynamic weighting optimization subproblem (cf. Eq. ( 5 )) guarantees Pareto stationarity in MOBL problems. Moreover, we establish a general result that the preference-guided dynamic weighted hypergradient direction in \mathsf{WC} - \mathsf{MHGD} satisfies a key minimum-norm condition (cf. Lemma 5.4 ), which serves as a foundation for our theoretical analysis and could be of independent interest for other MOBL research problems. • In addition to \mathsf{WC} - \mathsf{MHGD} algorithmic design and theoretical convergence analysis, we also offer the rationale behind our key algorithmic approaches and the geometric interpretation of our \mathsf{WC} - \mathsf{MHGD} approach in this paper, which advances and deepens our fundamental understandings of MOBL optimization theory and algorithms. Moreover, in Section 6 , we conduct extensive numerical experiments to verify our \mathsf{WC} - \mathsf{MHGD} algorithms. All of our empirical results confirm the efficacy and effectiveness of our \mathsf{WC} - \mathsf{MHGD} algorithms. 2 Related Work In this section, we provide an overview on recent works in MOBL to put our work in comparative perspectives. Due to space limitation, we relegate the overview of two closely related research

Read the complete text