CoAction: Cross-task Correlation-aware Pareto Set Learning
Xinyue Chen, Yingxuan Liang, Yiqin Huang, Chikai Shang, Hai-Lin Liu, Fangqing Gu ∗ · 2026
Read CoAction: Cross-task Correlation-aware Pareto Set Learning by Xinyue Chen, Yingxuan Liang, Yiqin Huang, Chikai Shang, Hai-Lin Liu, Fangqing Gu ∗ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01712.txt.
Introduction
Xinyue Chen and five co-authors teach one network to solve many multi-objective problems at once. Pareto set learning trains a model to turn any preference weighting into a matching best-compromise solution. The standard setup trains a fresh model per problem, wasting both compute and the kinship between tasks.
CoAction adds task awareness to a transformer backbone. Each task carries its own embedding, so the model can tell tasks apart while still sharing what it learns across them through self-attention. One trained model then serves the whole task family, reading a task tag plus a preference vector and returning the compromise. On broad multitask suites of benchmarks and real-world problems, the framework proves effective and competitive on the standard quality measures of volume covered, range, and spacing of solutions.
Catalog record
| CREATOR | Xinyue Chen, Yingxuan Liang, Yiqin Huang, Chikai Shang, Hai-Lin Liu, Fangqing Gu ∗ |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.01712
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CoAction: Cross-task Correlation-aware Pareto Set Learning Xinyue Chen, Yingxuan Liang, Yiqin Huang, Chikai Shang, Hai-Lin Liu, Fangqing Gu ∗ Abstract Pareto set learning (PSL) is an emerging paradigm in multi-objective optimization that trains neural networks to map preference vectors to Pareto optimal solutions. However, existing PSL methods primarily focus on solving a single multi-objective optimization problem at a time. This limitation not only increases computational costs in multi-objective multitask optimization scenarios by requiring a separate model for each task, but also fails to exploit the inter-task correlations across tasks. To address this, we propose a C r o ss-t A sk c orrela tion -aware Pareto Set Learning (CoAction) framework, which leverages task-aware transformer to handle multiple tasks simultaneously. Specifically, by assigning task-specific embedding vectors to individual tasks, the model effectively distinguishes between tasks while facilitating knowledge sharing among them. We utilize a Transformer encoder as the backbone architecture to leverage its self-attention mechanism for capturing complex task dependencies. The proposed approach is evaluated on comprehensive multitask test suites covering both benchmark problems and real-world applications, demonstrating effectiveness and competitive performance in Hypervolume, Range, and Sparsity. 1 Introduction Multi-objective optimization (MOO) aims to simultaneously optimize multiple conflicting objectives [ 7 , 18 ] , yielding a set of trade-off solutions known as the Pareto set (PS), whose image in objective space constitutes the Pareto front (PF) [ 31 ] . Many practical applications require Pareto optimal solutions to be generated in real time to support rapid switching between trade-off preferences. However, traditional evolutionary methods such as NSGA-II [ 6 ] , MO-CMA-ES [ 11 ] , SMS-EMOA [ 3 ] , and MOEA/D [ 28 ] produce only a finite number of optimal solutions per run and cannot respond to arbitrary preference without restarting the search, hindering real-time decision-making. To address this limitation, Lin et al. [ 13 ] proposed Pareto set learning (PSL), which trains a neural network to approximate the entire PS conditioned on preference vectors, enabling real-time generation of Pareto optimal solutions for any given preference. Despite these advances, a fundamental limitation of existing PSL methods is that they are designed to solve one optimization problem at a time, requiring an independent model to be trained for each problem. This paradigm incurs a computational cost that scales linearly with the number of problems and prevents the model from capturing and exploiting shared structure across related tasks. Such limitations become especially critical in practical settings where a large number of related optimization problems arise simultaneously and must be addressed efficiently. A natural and practically important example of such settings is multitask learning (MTL) [ 5 , 30 ] , where multiple tasks are optimized jointly and their inherent shared structure offers an opportunity for cross-task knowledge sharing. Sener and Koltun [ 21 ] formally cast the joint optimization of multiple tasks as a MOO problem, providing a principled theoretical foundation that motivates the application and extension of PSL methods to MTL scenarios. Extending PSL to handle multiple optimization problems simultaneously exposes several critical challenges. First, existing PSL methods treat each optimization problem independently, requiring a separate model per task and leading to significant computational inefficiency that scales linearly with the number of tasks [ 29 , 15 , 14 ] . Moreover, this independent treatment fundamentally fails to exploit inter-task relationships, leaving the latent structural similarities across different optimization problems untapped and losing the regularization and generalization that shared representations could provide. These limitations motivate a unified framework that shares parameters across tasks while explicitly modeling inter-task relationships, enabling both scalability and improved generalization. To address these challenges, we propose a Cross-task Correlation-aware Pareto Set Learning (CoAction) framework that learns Pareto optimal solution sets across multiple optimization problems within a single model. CoAction assigns unique task-specific embedding vectors to each optimization problem and employs explicit parameter sharing, enabling the model to distinguish between tasks and achieve cross-task knowledge sharing, thereby reducing computational costs. To explicitly model inter-task relationships, we adopt a Transformer encoder [ 10 ] as the backbone, whose self-attention mechanism captures complex global dependencies among tasks [ 24 ] and supports scalable parallel PSL. The main contributions of this paper are as follows. • We propose a CoAction framework with a Transformer encoder backbone that leverages task embedding, shared network parameters, and self-attention mechanism to model inter-task relationships. This enables a single model to learn Pareto optimal solution sets for multiple optimization problems simultaneously, reducing computational costs while enhancing model expressiveness. • We conduct extensive experiments on multitask test suites comprising benchmark problems and real-world engineering applications, demonstrating that the CoAction framework achieves competitive performance on hypervolume, range, and sparsity metrics. The remainder of this paper is organized as follows. Section 2 introduces the preliminaries of MOO, MTL, and the Transformer architecture. Section 3 presents the proposed CoAction framework, including the task embedding scheme and the multitask collaborative training algorithm. Section 4 reports comprehensive experimental results on benchmark and real-world problems, along with ablation studies, extended evaluations, and statistical analyses. Finally, Section 5 concludes the paper. 2 Preliminaries This section introduces the necessary background for the proposed method, covering MOO, MTL, and the Transformer architecture. 2.1 Multi-objective Optimization A MOO problem seeks to simultaneously minimize m\geq 2 objective functions over a decision space \mathcal{X}\subseteq\mathbb{R}^{n} : \min_{\mathbf{x}\in\mathcal{X}}F(\mathbf{x})=(f_{1}(\mathbf{x}),f_{2}(\mathbf{x}),\ldots,f_{m}(\mathbf{x})), (1) where \mathcal{Y}=F(\mathcal{X})\subseteq\mathbb{R}^{m} denotes the image of \mathcal{X} in the objective space. Definition 1 Pareto Dominance. A solution \mathbf{x}^{(a)} dominates \mathbf{x}^{(b)} , denoted \mathbf{x}^{(a)}\prec\mathbf{x}^{(b)} , if f_{i}(\mathbf{x}^{(a)})\leq f_{i}(\mathbf{x}^{(b)}) for all i\in\{1,2,\ldots,m\} and f_{j}(\mathbf{x}^{(a)})<f_{j}(\mathbf{x}^{(b)}) for at least one j\in\{1,2,\ldots,m\} . Definition 2 Pareto Set and Pareto Front. A solution \mathbf{x}^{*}\in\mathcal{X} is called a Pareto optimal solution if there exists no other solution that dominates it. The set of all Pareto optimal solutions is called the PS , and its image in objective space, \mathcal{F}:=F(PS) , is called the PF . Definition 3 Hypervolume Indicator. Given a reference vector \mathbf{r}\succeq\mathbf{y}^{\mathrm{nadir}} , with \mathbf{y}^{\mathrm{nadir}}=\max_{\mathbf{y}\in\mathrm{PF}}\mathbf{y} and \mathbf{y}^{\mathrm{ideal}}=\min_{\mathbf{y}\in\mathrm{PF}}\mathbf{y} defined component-wise. The Hypervolume (HV) indicator of a solution set A is defined as: \mathcal{H}_{\mathbf{r}}(A):=\Lambda\!\left(\bigcup_{\mathbf{p}\in A}\left\{\mathbf{q}\;\middle|\;\mathbf{p}\preceq\mathbf{q}\preceq\mathbf{r}\right\}\right), (2) where \Lambda(\cdot) denotes the Lebesgue measure. The HV quantifies the volume of the objective space dominated by A and bounded by \mathbf{r} , and serves as a standard quality indicator for evaluating the approximated PF. 2.2 Multitask Learning MTL improves generalization by jointly training a single model on multiple related tasks, allowing shared statistical strength to benefit each individual task [ 5 , 30 ] . Architecturally, MTL methods fall into two broad paradigms. Hard parameter sharing employs a common backbone with task-specific heads, acting as an implicit regularizer [ 23 , 2 ] , while soft parameter sharing maintains per-task parameters with explicit cross-task interaction [ 19 ] . Beyond architectural choices, MTL has been formally cast as a MOO problem [ 21 ] . This formulation has inspired PSL-based approaches that condition a single model on preference vectors, enabling scalable inference of diverse trade-off solutions for MTL [ 20 , 16 ] . 2.3 Transformer Architecture The Transformer [ 24 ] relies on the self-attention mechanism to capture global dependencies across the input sequence. Given input embeddings \mathbf{X}\in\mathbb{R}^{L\times d} , the attention output is computed as follows: \mathrm{Attention}(\mathbf{Q},\mathbf{K},\mathbf{V})=\mathrm{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^{\top}}{\sqrt{d_{k}}}\right)\mathbf{V}, (3) where \mathbf{Q} , \mathbf{K} , and \mathbf{V} denote the