Deep Learning for Subspace Regression
Vladimir Fanaskov, Vladislav Trifonov, Alexander Rudikov, Ekaterina Muravleva, Ivan Oseledets · 2025
Read Deep Learning for Subspace Regression by Vladimir Fanaskov, Vladislav Trifonov, Alexander Rudikov, Ekaterina Muravleva, Ivan Oseledets (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23249.txt.
Introduction
Reduced models need a subspace for each new parameter value, and grids fail in high dimensions. Vladimir Fanaskov and 4 co-authors swap interpolation for neural regression on subspaces. Several loss functions suit subspace data on the Grassmann manifold. One trick eases the learning a lot. The network predicts a larger subspace than the task strictly needs.
Theory shows that slack smooths the target map for elliptic eigenproblems. Empirically, accuracy rises sharply with the larger target. Demos span parametric eigenproblems, deflation, relaxation, control, and PDEs. Subspace regression thus becomes a practical online stage for reduced modeling.
Catalog record
| CREATOR | Vladimir Fanaskov, Vladislav Trifonov, Alexander Rudikov, Ekaterina Muravleva, Ivan Oseledets |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 84 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-23249
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Deep Learning for Subspace Regression Vladimir Fanaskov, Vladislav Trifonov, Alexander Rudikov, Ekaterina Muravleva, Ivan Oseledets Abstract It is often possible to perform reduced order modelling by specifying linear subspace which accurately captures the dynamics of the system. This approach becomes especially appealing when linear subspace explicitly depends on parameters of the problem. A practical way to apply such a scheme is to compute subspaces for a selected set of parameters in the computationally demanding offline stage and in the online stage approximate subspace for unknown parameters by interpolation. For realistic problems the space of parameters is high dimensional, which renders classical interpolation strategies infeasible or unreliable. We propose to relax the interpolation problem to regression, introduce several loss functions suitable for subspace data, and use a neural network as an approximation to high-dimensional target function. To further simplify a learning problem we introduce redundancy: in place of predicting subspace of a given dimension we predict larger subspace. We show theoretically that this strategy decreases the complexity of the mapping for elliptic eigenproblems with constant coefficients and makes the mapping smoother for general smooth function on the Grassmann manifold. Empirical results also show that accuracy significantly improves when larger-than-needed subspaces are predicted. With the set of numerical illustrations we demonstrate that subspace regression can be useful for a range of tasks including parametric eigenproblems, deflation techniques, relaxation methods, optimal control and solution of parametric partial differential equations. 1 Introduction The goal of reduced order modelling (ROM) is to identify uninformative degrees of freedom and discard them [ 3 ] . The result is a simplified system that is easier to analyse and simulate. This program is computationally demanding and only justified in the setting when many related problems are repeatedly solved and it is possible to use information from encountered problems to build a reduced model for the problems to come. Typical examples are parametric models for partial and ordinary differential equations (PDEs and ODEs), and usual applications are optimisation, sensitivity analysis, uncertainty quantification and control. As an illustration consider proper orthogonal decomposition (POD) for time-dependent PDEs [ 34 ] , [ 16 ] . To apply POD, one computes solutions for a representative set of parameters and builds a reduced basis for spatial variables by the best low-rank approximation. When new parameters arrive, a computed basis is used to discretise PDE that is solved at reduced cost. In global POD this basis is the same for all incoming parameters and in local POD basis explicitly depends on new parameters. As one may expect, local POD is more expressive than global POD, but can be more challenging to arrange. POD is an example of the general class of techniques where linear subspace parametrises useful degrees of freedom. While nonlinear ROM techniques exist, linear methods are better understood theoretically, easier to apply in practice, and provide sufficiently well approximation, especially when local versions are available [ 13 ] . In this setting the main challenge is to construct reliable approximation to the function that maps new parameters to linear subspaces. We analyse this problem under the following assumptions: (i) the set of parameters of interest is specified in form of probability distribution, (ii) it is known how to compute good or optimal linear subspace for each parameter, (iii) the numerically stable method to construct reduced problem from basis is available. In short, we consider regression on grassmannian. We approach the regression problem by specifying loss function and using neural networks as parametric models to accommodate high-dimensional parameter spaces pervasive in practical problems. More specifically, our main contributions are: 1. Mathematical formulation of subspace regression problem and examples of applications, including eigenspace approximation, local POD, learning basis for deflation and two-grid method, approximating balanced-truncation basis for optimal control problems. 2. Several loss functions, suitable for neural network training, including the stochastic one that scales well with the increase of subspace size. 3. Embedding technique: a strategy to learn a larger subspace containing the target one. Empirically, this technique significantly improves accuracy for subspace learning. 4. Two theoretical justification of embedding technique: derivative of smooth function on Grassmann manifold can be reduced by embedding; complexity of the subspace regression problem for elliptic eigenproblem with constant coefficients. 5. Empirical evaluation of proposed techniques on a diverse set of problems including comparisons with neural surrogates, kernel methods and classical interpolation in normal coordinates. Code used for training, evaluation and dataset generation is available at https://github.com/VLSF/subreg . 2 Subspace regression In this section we formulate precisely what we mean by subspace regression and describe several applications. 2.1 Definition of subspace regression problem In linear space \mathbb{R}^{n} we define k -dimensional subspace \mathcal{S}(W)=\left\{W\alpha,\alpha\in\mathbb{R}^{k}\right\} by specifying tall full rank matrix W\in\mathbb{R}^{n\times k} . Matrices W_{1} and W_{2} represent the same subspace if there is an invertible matrix G such that W_{1}=W_{2}G . The equivalence class of such matrices is denoted by \lceil W\rceil . The set of all k -dimensional subspace of n -dimensional space \text{Gr}(k,n) is known as Grassmann manifold or grassmanian [ 11 ] , [ 5 ] . Let V(r):\mathbb{R}^{p}\rightarrow\text{Gr}(k,n) be a function that maps the space of parameters r\in\mathbb{R}^{p} to the subset of grassmanian represented as the set of tall full rank matrices V(r) . We assume that parameters r are sampled from distribution r\sim p_{r} and that dataset \mathcal{D}=\left\{\left(r_{1},V_{1}\right),\dots,\left(r_{m},V_{m}\right)\right\} of m i.i.d. samples is available. For a given parametric model W_{\theta}:\mathbb{R}^{p}\rightarrow\text{Gr}(r,n),r\geq k we want to identify parameters \theta^{\star} such that W_{\theta^{\star}}(r) approximates V(r) . We formulate this task as optimisation problem \theta^{\star}=\arg\min_{\theta}\left(\underset{r\sim p_{r}}{\mathbb{E}}\left[L\left(W_{\theta}(r),V(r)\right)\right]\right)\simeq\arg\min_{\theta}\left(\frac{1}{m}\sum_{i=1}^{m}L\left(W_{\theta}(r_{i}),V_{i}\right)\right). (1) Loss function for subspace regression problem is assumed to have two properties: \begin{split}&L(A,B)=L(\widetilde{A},\widetilde{B})\text{ for arbitrary }\widetilde{A}\in\lceil A\rceil,\,\widetilde{B}\in\lceil B\rceil;\\ &L(A,B)>0\text{ and }L(A,B)=0\text{ iff }\mathcal{S}(B)\subset\mathcal{S}(A).\end{split} (2) In Section 3 we provide explicit expression for loss functions with these properties. Aside from unusual invariance requirement ( 2 ), optimisation problem ( 1 ) is a standard machine learning formulation of regression problems which can be solved with stochastic optimisation for arbitrary model W_{\theta}(r) that admits efficient evaluation of gradients. 2.2 Examples of subspace regression problem Approximate eigenspaces. Consider eigenproblem for Schrödinger equation -\Delta\psi(x)+U(x)=E\psi(x),\,\left\|\psi\right\|_{2}<\infty. (3) One way to find eigenpairs is to approximates \psi(x) by a finite series \psi(x)=\sum_{i=1}^{K}\alpha_{i}\phi_{i}(x) and enforce Petrov-Galerkin condition that residual is orthogonal to all \phi_{i}(x) . Continuous problem ( 3 ) reduces to eigenproblem for Hermitian matrix and can be solved in O\left(K^{3}\right) operations [ 32 ] . For eigenproblems we are typically interested only in extremal eigenspaces corresponding to either smallest or largest eigenvalues. In this case it is desirable to select a small number of basis functions \phi_{i}(x) that approximate sufficiently well the subspace of interest. When eigenproblem ( 3 ) is solver repeatedly for many potential functions U(x) this lead us to subspace regression problem ( 1 ) used to approximate the mapping U(x)\rightarrow\left\{\sum_{i=1}^{K}\alpha_{i}\phi_{i}(x)=f,\alpha_{i}\in\mathbb{R}\right\} . 1 1 1 Suitable discretisation of parametrisation of both \phi and U is assumed. When this mapping is learned from a set of examples, eigenproblems for unobserved potentials U can be solved efficiently, since low-dimensional candidate subspace for eigenfunctions is available. Intrusive POD for time-dependent PDEs. As an example of time-dependent PDE we use Burgers equation \frac{\partial u(x,t)}{\partial t}+u(x,t)\frac{\partial u(x,t)}{\partial x}=\frac{\partial}{\partial x}\left(\nu(x)\frac{\partial u(x,t)}{\partial x}\right),\,u(0,t)=u(1,t)=0,\,u(x,0)=u_{0}(x). (4) One starts with spatial discretisation which