Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning

Adrien Weihs, Hayden Schaeffer · 2026

Read Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning by Adrien Weihs, Hayden Schaeffer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22724.txt.

Introduction

Weihs and Schaeffer study Multiple Neural Operators, an architecture for learning collections of operators across tasks that share representations. For broad classes of Lipschitz operator maps they derive near-optimal bounds on approximation and generalization, plus a curse of parametric complexity and matching minimax rates. The upshot is that shared representations add no extra cost: multi-task operator learning follows the identical scaling laws as single-operator learning, and a DeepONet variant matches those rates.

Catalog record

CREATORAdrien Weihs, Hayden Schaeffer
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE167736
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.22724

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning Adrien Weihs, Hayden Schaeffer Abstract We study the approximation and statistical complexity of learning collections of operators in a shared multi-task setting, with a focus on the Multiple Neural Operators (MNO) architecture. For broad classes of Lipschitz multiple operator maps, we derive near-optimal upper bounds for approximation and statistical generalization. On the lower-bound side, we establish a curse of parametric complexity and prove corresponding minimax rates. Together, these results show that shared representations across tasks do not increase the overall cost: multi-task operator learning follows the same scaling laws as single operator learning. We also compare MNO with a multi-task extension of DeepONet based on concatenated task inputs and show that, from a worst-case approximation-complexity perspective, both architectures satisfy essentially the same asymptotic rates. Keywords and phrases. Deep Neural Networks, Approximation Theory, Neural Scaling Laws, Multi-task Learning, Multi-Operator Learning, Operator Learning. Mathematics Subject Classification. 41A99, 68T07 1 Introduction We study the problem of learning collections of operators in a shared multi-task setting, referred to as multi-operator learning or multi-task operator learning [ 43 , 44 , 47 , 46 ] . The goal is to approximate multiple operator maps, that is, maps of the form G:W\longrightarrow\{G[\alpha]:U\to V\}_{\alpha\in W}, within a single neural network model. Here, W,U,V are function spaces, \alpha\in W identifies the operator or task, and for each fixed \alpha , the corresponding map G[\alpha] sends an input function u\in U to an output in V . This viewpoint extends the standard operator-learning problem [ 23 , 36 ] , in which one seeks to approximate a single operator between function spaces. It also arises naturally in a wide variety of applications, including parameterized kernel operators, solution operators of parameterized PDEs, and task-conditioned operator families or foundation models; see [ 46 , Section 2] for a broader discussion. Learning such maps is particularly challenging for several reasons. First, multiple operator/multi-task learning is “effectively higher-dimensional” than standard operator learning. Second, the dependence on the operator descriptor \alpha , the input function u , and the evaluation variable x may all be highly nonlinear. Third, these variables play different roles in the problem, and this structure should be taken into account in the design of the learning mechanism. Given their expressivity and flexibility, neural networks are a natural approximation class in this setting. However, the associated architectural search problem remains a central challenge. Specifically, one seeks network structures that are empirically effective while also admitting theoretical guarantees. A natural way to quantify such guarantees is through their scaling laws. These describe how a target error metric, for instance approximation accuracy or generalization error, depends on a notion of complexity, such as architectural size, sparsity, or the number of training samples. In this sense, scaling laws relate achievable performance to the resources available for learning. A recent architecture addressing both empirical performance and theoretical guarantees is the Multiple Neural Operators (MNO) architecture [ 47 , 46 ] . Concretely, for Lipschitz multiple operator maps G , it was shown in [ 47 ] that for every target accuracy \varepsilon>0 , there exists a MNO with explicit \varepsilon -dependent bounds on the depth, width, sparsity, and parameter magnitude (see Table 3 ) satisfying \|\mathrm{MNO}-G\|\leq\varepsilon. In particular, these constructive bounds imply an estimate of the form N_{\#}(\mathrm{MNO})\leq\varepsilon^{-\varepsilon^{-\varepsilon^{-1/d}}}, where N_{\#} denotes the total number of nonzero parameters and d>0 depends on the underlying function classes. These approximation-theoretic bounds were then used in [ 46 ] to derive statistical learning rates. In particular, one obtains a generalization bound scaling as \left(\frac{\log\log n_{\alpha}}{\log\log\log n_{\alpha}}\right)^{-2/d}, where n_{\alpha} denotes the number of sampled operators G[\alpha] used during training. To the best of our knowledge, these rates were the first of their kind for multiple operator learning. However, they also exhibit an additional exponential blow-up relative to the corresponding rates in standard operator learning (see Table 1 ), raising the possibility that multiple operator learning may be intrinsically more complex and might suffer from a specific curse of parametric complexity . In this paper, we demonstrate that the complexity of multi-task operator learning matches the qualitative exponential rates one expects for single-task operator learning. Specifically, our first main result is a substantial improvement of the previously known approximation and generalization scaling laws for multi-task learning using MNO. In particular, we show that the additional constructive blow-up is not intrinsic: one can derive complexity upper bounds for MNO that match the scale of the corresponding operator-learning rates. This immediately yields a stronger bound in the statistical learning rates. Our second main contribution is a lower-bound theory for multiple operator learning. We prove lower complexity bounds for broad classes of Lipschitz and differentiable multiple operator maps, showing that the upper bounds obtained in this work are close to sharp and that some form of parametric complexity barrier is indeed unavoidable. Lastly, we show that the same minimax scaling laws also apply to alternative architectural approaches to multiple operator learning. In this way, the paper clarifies which aspects of the previously observed complexity growth are artifacts of the constructive analysis and which reflect intrinsic barriers of the target class. 1.1 Main Contributions Our main contributions are summarized as follows. 1. Near-optimal constructive approximation rates for MNO We derive near-optimal approximation-theoretic upper bounds for MNO on classes of Lipschitz multiple operator maps in Theorem 3.1 . The rates are obtained through a refined error analysis of the constructive approximation scheme. In particular, for every target accuracy \varepsilon>0 , we construct a MNO approximator with uniform error at most \varepsilon and with explicit \varepsilon -dependent bounds on the depth, width, sparsity, and parameter magnitude of its subnetworks. The precise scalings are given in Table 3 . The resulting total approximation complexity satisfies N_{\#}\lesssim\exp\left(d\log(\varepsilon^{-1})\,\varepsilon^{-\max\{d_{W},d_{U}\}}\right), where d_{W},d_{U} denote the dimension of the domain of functions in W,U respectively and d>0 depends on U and W . Inverting the relationship, we obtain the following scaling of error \varepsilon as a function of N_{\#} : \varepsilon\lesssim\left(\frac{\log N_{\#}}{\log\log N_{\#}}\right)^{-1/\max\{d_{W},d_{U}\}}. These rates are of iterated exponential-type, qualitatively matching the known rates for operator learning [ 31 , 26 , 42 , 38 , 15 , 25 , 37 , 24 ] . 2. Refined statistical learning rates for MNO We show that the refined approximation-theoretic bounds propagate directly to the statistical setting. More precisely, by combining the refined approximation construction with the generalization framework of [ 46 ] , we obtain stronger learning rates for MNO on Lipschitz multiple operator maps in Theorem 3.4 . In particular, the resulting generalization bound scales as \left(\frac{\log n_{\alpha}}{\log\log n_{\alpha}}\right)^{-2/\max\{d_{W},d_{U}\}}, thus improving the previously known rates and reducing them to the corresponding operator-learning scale. 3. Lower complexity bounds for multiple operator learning We extend the lower-complexity framework of [ 26 ] to the multiple operator setting. We introduce the notion of multiple operator maps of neural network type adapted to separable architectures such as MNO. As a consequence, in Theorem 3.8 and Lemma 3.10 , we obtain lower complexity bounds for broad classes of Lipschitz and differentiable multiple operator maps, proving the same curse of parametric complexity as in operator learning for the multiple operator case. Specifically, we show that there exists a multiple operator map

Read the complete text