Transformer Causality Regularization for Dynamic Inverse Problems

Gesa Sarnighausen, Anne Wald, Andreas Hauptmann · 2026

Read Transformer Causality Regularization for Dynamic Inverse Problems by Gesa Sarnighausen, Anne Wald, Andreas Hauptmann (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19847.txt.

Introduction

Sarnighausen, Wald, and Hauptmann add a causality rule to dynamic inverse problems. The rule says a state at one time can depend only on past states, never on future ones. They use a transformer with a causal mask to learn that past-to-present map. A convolutional network handles the spatial part of each image. The transformer output then acts as a prior in a classic variational method. Tests on dynamic CT, with simulated and real data, beat earlier regularizers.

Catalog record

CREATORGesa Sarnighausen, Anne Wald, Andreas Hauptmann
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55545
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.19847

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Transformer Causality Regularization for Dynamic Inverse Problems Gesa Sarnighausen, Anne Wald, Andreas Hauptmann Abstract We study the concept of including the causality principle as regularizer into the solution of linear time-dependent inverse problems. This is achieved by combining transformer-based predictions with classical variational regularization, resulting in what we call transformer causality regularization (TCR). The causality principle states that an object at time t ′ t^{\prime} depends only on its previous states at t < t ′ t<t^{\prime} and is independent of future states at t > t ′ t>t^{\prime} . Since the transformer architecture represents sequence-to-sequence functions and can be equipped with a causal attention mask, transformers are the natural choice for a learned causality function that predicts the state of an object at time t ′ t^{\prime} given the previous states at t < t ′ t<t^{\prime} . We combine this with the inductive bias of convolutional neural networks (CNNs) for imaging tasks to treat the spatial variable. The output of the spatial-temporal transformer is then used as a prior for variational regularization, such that classical results on regularization and convergence for solution methods directly transfer to our case. Using the example of dynamic computerized tomography, we compare TCR to a static and dynamic version of the earlier introduced unrolled adversarial regularizer for simulated and measured data. The results show that using TCR within a variational framework improves reconstruction results and data-consistency. I Introduction Dynamic or time-dependent inverse problems (DIPs) arise in many applications where the quantity of interest evolves over time and is reconstructed from indirect observations. Prominent examples include imaging of process flows [ 23 ] as well as the estimation of time-dependent parameters in partial differential equations [ 31 ] . Here, we consider linear dynamic inverse problem, which can be expressed by the operator equation \displaystyle A(\vartheta)=\psi,\quad A:\mathbb{X}\to\mathbb{Y}, where the linear forward operator A , the target \vartheta and the data \psi are time- and space-dependent. The spaces \mathbb{X} and \mathbb{Y} are function spaces reflecting the time-dependence of the inverse problem, see, e.g., [ 9 , 51 ] . In this article, we consider the case of reconstructing the moving object instead of a fixed state. This means that the problem is heavily under-sampled since in general there are only a few measurements available per time step. Research on dynamic inverse problems is often linked to a concrete application, see [ 30 , 32 , 26 ] , such as dynamic computerized tomography [ 8 , 20 , 19 , 41 , 21 ] , magnetic resonance imaging [ 27 , 34 ] , emission tomography [ 24 ] , magnetic particle imaging [ 3 , 29 ] , or structural health monitoring [ 32 , 35 , 54 ] ; In many cases, additional information on the motion is included to reduce the degrees of freedom in a meaningful way [ 19 , 8 , 16 , 18 , 7 , 47 , 15 ] . We aim to develop methods that do not depend on the specific forward operator and do not assume prior knowledge on the motion. In recent years, data-driven approaches have become increasingly popular for solving (static) inverse problems [ 4 ] . A variety of approaches have been proposed to incorporate neural networks into the reconstruction process, ranging from purely data-driven methods to hybrid schemes that combine learned components with classical model-based regularization techniques [ 22 , 45 ] . One possibility is to extend the classical method of variational regularization by learning a regularization or penalty term [ 40 , 43 , 44 , 33 , 36 , 28 ] . In classical variational regularization this term needs to be handcrafted and depends on a-priori assumptions on the solution. In this study, we propose to learn a regularizer using transformer-based architectures. Transformers were originally introduced for language processing tasks [ 57 ] and are the backbone of many foundation models. In image processing tasks, transformers are primarily used for inpainting, denoising, or segmentation [ 37 , 59 , 60 , 49 ] . With the growing popularity of transformer-based architectures, transformers have also been used in approaches for solving inverse problems, e.g., for specific static inverse problems such as inverse scattering (post-processing) [ 12 ] or PDE-based inverse problems [ 17 ] . In [ 2 ] the authors solve different static linear inverse problems with in-context learning using transformers. Transformers have also been employed for solving dynamic inverse problems. In [ 46 ] dynamic radar inverse problems are solved using a spatial-temporal transformer that can capture the temporal dependencies. The authors in [ 53 ] trained a spatial-temporal transformer for dynamic scattering media video reconstructions. The aim of this article is to extend the research on classical solution methods for dynamic inverse problems in Lebesgue-Bochner spaces [ 51 , 9 ] , which are the usual choice for time-dependent partial differential equations, to the deep learning framework and solve dynamic inverse problems using data-driven regularization methods with regularization guarantees. In particular, this is achieved by encoding the concept of causality in a spatial-temporal transformer prediction model using causal attention masks. The sequence-to-sequence nature of transformers is particularly suited to deal with a varying length of the temporal information. The transformer predictions are then used to solve a classic variational problem to enforce data-consistency over long time periods. The resulting model is termed Transformer Causality Regularization (TCR), see Sections II-A and III . We compare our proposed model to the unrolled adversarial regularizer (UAR) [ 43 ] , a data-driven method developed for static inverse problems, Section II-B , which we extend to a dynamic setting. To provide a baseline on learned regularization for dynamic inverse problems, we first apply the UAR to a dynamic and static setting in Section II-B without any information on the motion analogously to Tikhonov regularization in Bochner spaces in [ 51 ] . In both cases, classical and data-driven, we understand the object of interest as a time-dependent function. In particular, this means that in the dynamic setting of UAR the measured data is available on the whole spatial-temporal domain. In the classical dynamic setting, however, the difference between time and space is expressed by different respective regularities [ 51 ] , as is usual in dynamic compressed sensing magnetic resonance imaging [ 56 ] , i.e., different regularization functionals are chosen for time and space. Nevertheless, in the UAR approach, the difference between time and space is not manually determined in advance but learned by the network. Since the solution is computed for all time points simultaneously in the dynamic setting, the data from later time points can also influence the reconstructions from earlier time points. This assumption can for some applications contradict the nature of time and the principle of causality, which says that the state of an object \vartheta\in\mathbb{X} at time t^{\prime} , i.e., \vartheta(t^{\prime}) , depends on its previous states \vartheta(t) for t<t^{\prime} only and cannot be influenced by future states. If we are, for example, interested in reconstructing a moving object in real-time, it would not be feasible to only get the reconstruction after the movement is finished. Such a causality principle can be implemented through a

Read the complete text