Fast Large-Scale Model-Based Iterative Tomography via Exploiting Mathematical Structure, Hierarchical Optimization, Smart Initialization, and Distributed GPU Computing Dinesh Kumar, Jeffrey J. Donatelli Abstract Model-Based Iterative Reconstruction (MBIR) is important because direct methods, such as Filtered Back-Projection (FBP) can introduce significant noise and artifacts in sparse-angle tomography, especially for time-evolving samples. Although MBIR produces high-quality reconstructions through prior-informed optimization, its computational cost has traditionally limited its broader adoption. In previous work, we addressed this limitation by expressing the Radon transform and its adjoint using non-uniform fast Fourier transforms (NUFFTs), reducing computational complexity relative to conventional projection-based methods. We further accelerated computation by employing a multi-GPU system for parallel processing. In this work, we further accelerate our Fourier-domain framework, by introducing four main strategies: (1) a reformulation of the MBIR forward and adjoint operators that exploits their multi-level Toeplitz structure for efficient Fourier-domain computation; (2) an improved initialization strategy that uses back-projected data filtered with a standard ramp filter as the starting estimate; (3) a hierarchical multi-resolution reconstruction approach that first solves the problem on coarse grids and progressively transitions to finer grids using Lanczos interpolation; and (4) a distributed-memory implementation using MPI that enables near-linear scaling on large high-performance computing (HPC) systems. Together, these innovations significantly reduce iteration counts, improve parallel efficiency, and make high-quality MBIR reconstruction practical for large-scale tomographic imaging. These advances open the door to near-real-time MBIR for applications such as in situ, in operando, and time-evolving experiments. 1 Introduction Synchrotron micro- and nano-computed tomography (micro-CT and nano-CT) [ 14 , 18 , 7 ] have become indispensable tools across a wide range of scientific fields, including materials science, geoscience, and energy research. These techniques enable non-destructive three-dimensional imaging of the internal structure of materials with high spatial resolution. Recent advances in synchrotron instrumentation and detector technology have significantly increased acquisition speeds, enabling new experimental modalities such as time-resolved tomography. Time-resolved tomography allows researchers to observe the evolution of microstructural processes within materials. Examples include crack propagation under mechanical stress [ 13 ] , degradation of polymer electrolytes during battery operation [ 11 ] , and fluid transport through porous media [ 1 ] . Capturing these dynamic processes requires rapid acquisition of projection data, often under strict dose or time constraints. As a result, the collected datasets frequently contain limited angular sampling and increased measurement noise, making accurate image reconstruction significantly more challenging. Another application that would benefit from MBIR is laminography which is inherently limited-angle with a missing-wedge because of sample geometry [ 15 ] . Model-based iterative reconstruction (MBIR) has emerged as a powerful framework for addressing these challenges. By incorporating prior knowledge and accurate physical models into the reconstruction process, MBIR can produce higher-quality reconstructions from noisy or incomplete data compared to conventional analytical approaches [ 20 , 16 , 12 ] . However, these benefits come at the cost of substantial computational expense. Traditional MBIR implementations rely on ray-tracing methods for evaluations of forward and adjoint projection operators, which makes reconstruction prohibitively slow for large-scale or time-resolved tomography experiments. To address this limitation, the TomoCAM framework [ 12 ] was developed as a GPU-accelerated platform for tomographic reconstruction. TomoCAM implements the Radon transform and its adjoint using non-uniform fast Fourier transforms (NUFFTs) [ 9 , 5 ] , enabling significantly faster computation compared to conventional ray-tracing methods. In addition, the framework leverages multi-GPU architectures to parallelize the reconstruction process, allowing large datasets to be processed more efficiently. In this work, we present several strategies that further improve the performance and scalability of the TomoCAM framework. The main contributions of this work are: • Reformulating the gradient operator using a multi-level Toeplitz structure enables the use of FFT-based fast convolutions • A hierarchical multi-resolution optimization strategy that accelerates convergence by progressively refining the reconstruction grid. • A filtered backprojection-based initialization that significantly reduces the number of iterations required for convergence. • A hybrid distributed MPI and multi-GPU implementation that enables large-scale tomographic reconstruction on modern HPC systems. The code is available under an open-source license at https://github.com/lbl-camera/tomocam . 2 Theory and Mathematical Background Tomographic reconstruction is fundamentally based on the inversion of the Radon transform. We briefly review the mathematical background here and refer the reader to standard texts for additional details [ 3 , 10 ] . Let f:\mathbb{R}^{2}\rightarrow\mathbb{R} . The Radon transform of f is defined by line-integral R_{\theta}\,f(t)=\int_{\ell(t,\theta)}f(x,y)d\ell (1) where \ell(t,\theta) denotes the line parameterized by the offset t and projection angle \theta . The Radon transform is closely related to the Central Slice Theorem , which states that the one-dimensional Fourier transform of a projection acquired at angle \theta corresponds to a slice of the two-dimensional Fourier transform of the object f(x,y) taken at the same angle \theta , i.e., \mathbb{F}R\,f(\omega)=\hat{f}(\omega\cos\theta,\omega\sin\theta) (2) where \hat{f} in two-dimensional Fourier transform of f , and (\omega\cos\theta,\omega\sin\theta)=(k_{x},k_{y}) the Fourier-domain coordinates. Traditional iterative tomographic reconstruction methods often rely on repeatedly applying forward and back-projection operators, which can be interpreted as large implicit linear systems. In previous work [ 12 ] , we implemented these operators using non-uniform fast Fourier transforms (NUFFTs), reducing the computational complexity from O(N^{2}) to O(N\log N+M\log_{10}\epsilon^{-1}) , where N denotes the number of voxels in target reconstruction, M is the total number of pixels in the projection data, and \epsilon is the desired numerical precision. The forward projection, i.e., the output of the Radon transform can be calculated as Rf=\mathbb{F}^{-1}\mathcal{F}f, (3) where \mathcal{F} denotes the NUFFT Type-2 operation that maps the Cartesian grid in the object domain to a polar grid in the Fourier domain, and \mathbb{F}^{-1} denotes the one-dimensional inverse FFT applied along the radial direction for a given projection angle. Similarly, the adjoint operation, commonly referred to as back-projection, can be expressed as R^{*}g=\mathcal{F}^{*}\mathbb{F}g, (4) where \mathcal{F}^{*} denotes the NUFFT Type-1 operation that maps the polar grid in the Fourier domain back to the Cartesian grid in the object domain, and \mathbb{F} denotes the one-dimensional FFT along the radial direction. When the projection data doesn’t satisfy angular Shannon-Nyquist criterion, regularization \Phi is required. The tomographic reconstruction problem can then be formulated as the following optimization problem: \hat{f}=\arg\min_{f}\frac{1}{2}\|Rf-g\|_{2}^{2}+\lambda\Phi(|\nabla f|). (5) Here, g denotes the measured projection data, \Phi is a regularization function that enforces prior knowledge, and \lambda is a regularization parameter that balances data fidelity and regularization. TomoCAM uses an edge-preserving qGGMRF prior for \Phi ; we refer the reader to [ 20 , 16 ] for details. The formula for the iterative step is obtained by differentiating ( 5 ), i.e., f^{k+1}=f^{k}-\frac{1}{L}\left(R^{*}(R\,f-g)-\lambda\nabla\Phi(|\nabla f|)\right), (6) where L is the Lipchitz constant of the gradient. It is worth noting that, for 3D reconstruction, the Radon-based data fidelity term is evaluated slice-by-slice, whereas the qGGMRF prior enforces spatial coupling across the full 3D volume. This optimization problem can be solved using various iterative algorithms. In this work, we employ the Fast Iterative Method with Restart [ 8 ] , an accelerated variant of Nesterov’s method [ 17 ] , due to its favorable convergence properties for large-scale problems. In the following sections, we describe several newly implemented strategies that further improve the convergence rate and computational efficiency of this framework. 2.1 Toeplitz Structure Exploitation The primary computational bottleneck in solving Eq. ( 5 ) arises from repeated evaluations of R^{*}(Rf-g) in ( 6 ), during gradient computation. Each evaluation requires two NUFFTs. Although NUFFTs have log-linear complexity, the gradient calculation can be further accelerated by exploiting the Toeplitz structure of operator composition R^{*}\,R . Specifically, this composition can be expressed as a convolution between f and a point spread function (PSF) kernel K , that depends on the coordinates sampled in the Fourier space. While Toeplitz acceleration has been explored in MRI reconstruction [ 4 , 19 ] and cryo-EM [ 21 ] , its application to NUFFT based tomographic formulations has received limited attention. This observation allows the data fidelity gradient to be computed using FFT-based convolution. The data fidelity gradient is given by \nabla_{f}e=R^{*}(Rf-g)=R^{*}Rf-R^{*}g. (7) In ( 7 ), we can rewrite R^{*}Rf=K\circledast f, (8) where \circledast refers to convolution. The PSF kernel K can be precomputed by applying \mathcal{F}^{*} to a function \mathbf{1} , where \mathbf{1}(x,y)=1 for all (x,y) sampled in Fourier space. To accurately recover R^{*}R\,f , the PSF kernel K must be sampled on a larger grid with at least 2N-1 grid points in each dimension. Accordingly, to compute the gradient, f is zero-padded to size 2N-1 in each dimension, and the resulting convolution is restricted to an N\times N region. The convolution can then be evaluated efficiently using FFTs: R^{*}Rf=Res_{N\times N}(\mathbb{F}^{-1}\left(\mathbb{F}K\odot\mathbb{F}Pad_{2N-1\times 2N-1}f\right), (9) where \mathbb{F} denotes the two-dimensional fast Fourier transform, Res is restriction, Pad is extension, and \odot represents element-wise multiplication. The reformulated loss and gradient equations are given as, \displaystyle e \displaystyle=\frac{1}{2}f^{\intercal}\,K\circledast f-f^{\intercal}\,R^{*}g+\frac{1}{2}g^{\intercal}g (10) \displaystyle\nabla_{f}e \displaystyle=K\circledast f-R^{*}g (11) Both \mathbb{F}K and R^{*}g are computed once prior to the iterative reconstruction. This reformulation significantly reduces the computational cost of the data fidelity gradient and loss, as it requires only two FFTs and one element-wise multiplication instead of a forward and a backprojection. 2.2 Initialization Strategy The choice of initialization can significantly influence the convergence behavior of iterative reconstruction algorithms. Starting the optimization from a poor initial estimate may require many iterations before the algorithm approaches a meaningful solution. To mitigate this issue, we adopt an informed initialization strategy based on filtered backprojection (FBP), which provides a fast approximate reconstruction from the measured projection data [ 3 , 10 ] . Let g denote the measured projection data and R the forward projection operator. A quick initial estimate can be obtained by applying the adjoint operator R^{*} to the projection data. However, an unfiltered backprojection produces a blurred object due to the frequency response of the Radon transform. To compensate for this effect, the projection data are first filtered with a ramp filter prior to backprojection [ 6 ] . Let p_{\theta}(s) denote the projection at angle \theta and sinogram coordinate s . The filtered projection is computed as \tilde{p}_{\theta}(s)=(h\circledast p_{\theta})(s), (12) where \circledast denotes convolution and h(s) is the ramp filter kernel. In the Fourier domain, the ramp filter is defined as \hat{h}(\omega)=|\omega|, (13) which amplifies high-frequency components to compensate for the smoothing introduced by the projection process. The initial guess is then obtained using filtered backprojection f_{0}=R^{*}\tilde{g}, (14) where \tilde{g} denotes the collection of filtered projections. This initialization provides a computationally inexpensive approximation of the true solution. Although the filtered backprojection reconstruction may contain artifacts due to limited-angle sampling or measurement noise, it typically captures the dominant low-frequency structure of the underlying image. Consequently, it serves as an effective starting point for the iterative optimization. By initializing the reconstruction with f_{0} , the algorithm begins closer to the desired solution, reducing the number of iterations required for convergence and lowering the overall computational cost. 2.3 Hierarchical Multi-Resolution Reconstruction To accelerate convergence, we employ a hierarchical multi-resolution reconstruction strategy. Instead of solving the inverse problem directly at the target resolution, the reconstruction is performed progressively across a sequence of grids with increasing spatial resolution. This approach leverages the observation that the low-frequency components of the solution dominate the early stages of reconstruction and can be estimated reliably on coarser grids at significantly lower computational cost. We define a hierarchy of grids \Omega_{0},\Omega_{1},\dots,\Omega_{L} , where \Omega_{0} corresponds to the coarsest discretization and \Omega_{L} to the desired reconstruction resolution. At each level l , the optimization problem in Eq. ( 5 ) becomes f^{(l)}=\arg\min_{f}\;\frac{1}{2}\|R_{(l)}f-g_{(l)}\|_{2}^{2}+\lambda\Phi(|\nabla f|), (15) where R_{(l)} denotes the forward operator defined on grid \Omega_{l} , and g_{(l)} represents projection data that have been downsampled to match that resolution. For downsampling, we select coarse-grid points by subsampling the full-resolution projection data. The optimization is first performed on the coarsest grid \Omega_{0} , producing an estimate f^{(0)} . After convergence at this level, the solution is interpolated to the next finer grid using Lanczos interpolation. Lanczos interpolation is based on a windowed sinc kernel that provides high-quality band-limited interpolation while suppressing ringing artifacts. Let P_{l} denote the interpolation operator mapping grid \Omega_{l} to \Omega_{l+1} . The initialization for the next level is given by f^{(l+1)}_{0}=P_{l}f^{(l)}. The Lanczos kernel with window parameter a is defined as L(x)=\begin{cases}\frac{\sin(\pi x)\,\sin(\pi ax)}{a\pi^{2}x^{2}},&|x|