Projected iterated Tikhonov regularization in low precision
C. Drum, J. G. Nagy, L. Onisk · 2025
Read Projected iterated Tikhonov regularization in low precision by C. Drum, J. G. Nagy, L. Onisk (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00669.txt.
Introduction
C Drum, J Nagy, and L Onisk wrote a 2025 paper in numerical math on solving inverse problems. Such problems work backward from noisy data to the hidden cause, and they are often unstable. Tikhonov methods calm that instability, and repeating them in steps can sharpen the answer.
The authors study a projected, repeated form of the method that works inside Krylov subspaces. Those are small, smart search spaces built step by step from the data. Their twist is to run the math in lower precision than usual double precision. Lower precision saves time and memory, so knowing whether the method still behaves well matters for large jobs.
Catalog record
| CREATOR | C. Drum, J. G. Nagy, L. Onisk |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-00669
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Projected iterated Tikhonov regularization in low precision C. Drum, J. G. Nagy, L. Onisk Abstract . We investigate the regularizing behavior of an iterative Krylov subspace method for the solution of linear inverse problems in precisions lower than double precision. Recent works have considered the projection of iterated Tikhonov methods using Krylov subspaces for both computational efficiency and an additional regularizing effect. To investigate the regularizing behavior of this projected algorithm applied to problems that are naturally severely ill-posed, we formulate the iterates as a filtered solution using the preconditioned Landweber method with a Tikhonov-type preconditioner in a Krylov subspace. Through numerical examples simulating multiple low precision choices, we showcase the filtering properties of the method and the achievement of comparable working accuracy applied to discrete inverse problems (i.e., to within a few decimal places in relative error) compared to results computed in traditional double precision. 1. Introduction Herein, we are interested in computing approximate solutions to linear least-squares problems of the form (1) \min_{x\in\mathbb{R}^{n}}\left\|Ax-b\right\|, in computing environments that support low precision and where data is often stored with less significant digits, e.g., medical tomography [ 31 ] , industrial applications [ 4 ] , and training for deep learning [ 14 ] . Here, and throughout we denote the Euclidean norm by \|\cdot\| . For some of these applications the singular values of the matrix A\in\mathbb{R}^{m\times n} can decay without significant gap and cluster at the origin (i.e., the matrix is ill-conditioned). These types of problems are commonly referred to as linear discrete ill-posed problems. Classically, they can arise through the discretization of Fredholm integral equations of the first kind [ 19 , 28 ] , but can also arise in modern large scale data problems such as, for example, the training of the random feature model in machine learning [ 38 ] . In applications, the vector b\in\mathbb{R}^{m} in ( 1 ) often represents observed measurements that are corrupted by error arising through noise contamination or instrumentation disturbances. Numerically, this error may also include truncation error or approximation error. We denote this error by e so that b=b_{true}+e where b_{true} is the unknown error-free vector associated with b . The goal when considering linear discrete ill-posed problems is to determine the solution of minimal Euclidean norm, x^{\dagger} , of the unavailable least-squares problem (2) x^{\dagger}=\arg\min_{x\in\mathbb{R}^{n}}\left\|Ax-\hat{b}\right\|. In the linear discrete inverse problem literature it’s well understood that because b is contaminated by error and the singular values of A cluster at the origin that the solution of ( 1 ) of minimal Euclidean norm is usually a poor approximation to x^{\dagger} . The common strategy is to regularize the problem - that is, to replace the problem ( 1 ) with a nearby problem that is less sensitive to the error in b . Recent progress in GPU hardware that supports floating point computations in mixed-to-low precision has enabled the numerical linear algebra community to develop robust decomposition strategies as well as accelerated iterative methods; see the surveys [ 1 , 30 ] . Iterative methods that utilize Krylov subspaces using one or more precisions lower than fp64 (i.e., double precision) have recently been considered in the literature for well-posed problems, see e.g., [ 11 , 12 , 13 ] . While some works including [ 2 ] and [ 23 ] have considered low or mixed precision in the inverse problem community, comparatively less has been studied, especially for iterative Krylov methods. Recent work by Li in [ 33 ] considered LSQR in low precision for linear inverse problems with the conclusion that ‘LSQR does not need to be solved in high precision’. However, the standard practice in many iterative Krylov methods today is to utilize a hybrid approach whereby one both projects the problem into a subspace and then performs regularization through a variational approach, rather than solely regularizing implicitly via subspace projection; see the review [ 17 ] . In a similar vein to the ‘project then regularize’ strategy just mentioned, methods based on projected iterated Tikhonov regularization with a nonstationary regularization parameter using a Krylov subspace have garnered recent attention; see [ 3 , 9 , 10 , 15 ] . Classically, iterated Tikhonov regularization is used to approximate the solution to ( 1 ) and at its k^{th} iterate is given by (3) x^{(k)}=x^{(k-1)}+\left(A^{T}A+\alpha^{2}I\right)^{-1}A^{T}r^{(k-1)} with the residual denoted by r^{(k-1)}=b-A^{(k-1)} ; see [ 8 , 25 ] . Here, \alpha>0 is the regularization parameter that balances the fidelity of the approximation ( 3 ) to ( 1 ). A major drawback to ( 3 ) is that if A is large then a large system of equations with the matrix A^{T}A+\alpha^{2}I has to be solved at each iteration. In [ 10 ] a low rank approximation to A at each step k using the Arnoldi process was used as an efficient alternative to the work by Donatelli and Hanke in [ 18 ] which replaced A with an approximation C in ( 3 ) whose structure allows for fast computation (e.g., using the discrete Fourier transform where appropriate). To generalize the work of [ 10 ] , the authors of [ 9 ] additionally considered the use of Golub-Kahan bidiagonalization (GKB) when both A and A^{T} are available and A is possibly non-square. Additionally, they investigated use of a precomputed GKB subspace, which was found to provide better accuracy in practice then the technique of iteratively increasing the subspace in [ 10 ] . This method is referred to as the projected iterated Tikhonov (PIT) method. Herein, we investigate the regularizing behavior of the PIT method applied to ( 1 ) in low precision with a nonstationary regularization parameter scheme. To do so, we formulate the iterates of PIT as a projected preconditioned Landweber method with a Tikhonov-type preconditioner that may be computed in a precision lower than fp64 using the GKB process. When considering a mixed precision approach to the Tikhonov problem in [ 35 ] , it was shown that the precision of the Tikhonov-type preconditioner used in the Landweber method controlled the efficacy of the solution. Alternatively, here we demonstrate that the efficacy of the solution in \mathbb{R}^{n} can be strongly dependent on the quality of the Krylov basis vectors which, in turn can depend on the working precision. Numerically, we illustrate that when computed in sufficiently low precision that the GKB process requires reorthogonalization to retain the same solution quality as compared to when the method is computed entirely in fp64. While works including [ 8 , 9 , 10 , 18 ] consider a nonstationary regularization parameter scheme utilizing a modified Levenberg-Marquardt iteration to solve a damped version of the discrepancy principle, we consider an alternative method based on a secant-update method using the discrepancy principle which requires fewer user chosen parameters and was first proposed in [ 21 ] for an iterative Krylov subspace method for linear discrete inverse problems. To gain insight into the effect that the nonstationary regularization parameters have on the PIT method, we derive both the stationary and nonstationary projected filter factors and confirm numerically that the nonstationary filter factors match those that are experimentally realized. We additionally demonstrate in our numerical results that