BART Streams: Real‐Time Reconstruction Using a Modular Framework for Pipeline Processing Authors: Philip Schaten, Moritz Blumenthal, Bernhard Rapp, Christina Unterberg‐Buchwald, Martin Uecker Published: 2026 ABSTRACT ABSTRACTPurposeTo create modular solutions for interactive real‐time MRI using reconstruction algorithms implemented in BART.MethodsA new protocol for streaming of multidimensional arrays is presented and integrated into BART. The new functionality is demonstrated using examples for cardiac interactive real‐time MRI based on radial FLASH, where iterative reconstruction is combined with advanced features such as dynamic coil compression and gradient‐delay correction. We analyze the latency of the reconstruction and measure end‐to‐end latency of the full imaging process.ResultsReconstruction pipelines with iterative reconstruction and advanced functionality were built in a modular way using scripting. Latency measurements demonstrate latency sufficient for interactive real‐time MRI, on the order of 30 ms for BART processing and network transfer time, or 200 ms for end‐to‐end latency including acquisition, vendor processing, and display.ConclusionWith the new streaming capabilities, real‐time reconstruction pipelines can be assembled using BART in a flexible way, enabling rapid prototyping of advanced applications such as interactive real‐time MRI. Introduction “Live streams” from inside the human body were possible for a long time [1] through the combination of real‐time MRI acquisition, real‐time reconstruction and low‐latency visualization. We refer to this as interactive real‐time MRI (RT‐MRI) [2, 3]. It is an important tool with emerging clinical applications, such as interventional [4, 5, 6, 7, 8] or fetal MRI [9]. RT‐MRI usually requires high frame rates, for example, to resolve cardiac motion [10], speech production [11, 12], or joint motion [13]. Thus, RT‐MRI often makes use of data undersampling in combination with specific reconstruction methods, for example view‐sharing [1], parallel imaging [14], compressed sensing [15, 16], Kalman filtering [17], spatiotemporally constrained reconstruction [18], or other. In this work, we show examples using the real‐time regularized non‐linear inversion (RT‐NLINV) algorithm [19, 20]. Based on classic regularized non‐linear inversion (NLINV) [21], RT‐NLINV offers particularly high undersampling factors and is causal, that is, it does not rely on data acquired after the frame that is currently being reconstructed. Running iterative RT‐MRI reconstruction algorithms such as RT‐NLINV in real‐time typically requires optimized software and hardware acceleration for example, through the use of Graphical Processing Units (GPU) [22, 23, 24], especially when low latency is required. For example, cardiac interventional MRI and MRI‐guided radiotherapy might require a latency as low as several 100 ms [7, 25], and speech biofeedback studies were performed with a latency of 100 ms [26]. The BART toolbox for computational MRI [27] is a comprehensive framework which provides a wide range of reconstruction, calibration [28], machine‐learning [29, 30], and signal processing tools. Its optimized code and its existing support for GPU computing make it an ideal choice for the development of real‐time reconstruction methods. Another core feature of BART is its modular design as a toolbox: Algorithms are made available as individual command line tools, enabling the construction of powerful reconstruction pipeline by combining the different tools in a script. However, high‐quality, low latency RT‐MRI requires not only optimized reconstruction, but also the ability to stream the data and reconstructed images during the measurement. Each tool in the reconstruction pipeline needs to perform its task as soon as a raw‐data frame is available, as opposed to waiting for a complete dataset. Existing software such as the Gadgetron [31], can stream data in ISMRMRD format [32], and BART has been used with Gadgetron in the past [33, 34]. However, up to now, it was not possible to use BART to construct low‐latency processing pipelines. In this work, we integrate new stream processing capabilities into BART. The new functionality facilitates building reconstruction pipelines suitable for RT‐MRI with existing BART tools. Importantly, our approach to streaming builds on standard Unix system utilities, which allows us to preserve the full modularity of BART. This is demonstrated using examples for interactive real‐time MRI based on radial FLASH, where iterative reconstruction is combined with advanced features such as dynamic coil compression and gradient‐delay correction. In particular, we show how the RT‐NLINV algorithm implemented in BART can now be used in real‐time reconstruction pipelines, and how a streaming version of geometric coil compression [35] acquires a new use as dynamic coil compression for interactive RT‐MRI. Additionally, we analyze the latency of the reconstruction itself and measure the end‐to‐end latency of the complete imaging process. Methods We will first give an overview over the architecture of BART, then discuss multi‐dimensional arrays, and then explain the new support for looping, and streaming, which is then used for real‐time processing. BART Structure BART is a modular reconstruction framework which enables building complex reconstruction algorithms through combination of various individual components. On the lowest level, a numeric library provides generic functions for operations on multidimensional arrays (md‐arrays). Many of these operations can be accelerated using GPUs, or with the Message Passing Interface (MPI) [36] toolkit, enabling distributed computing on scientific high‐performance clusters and multi‐GPU usage [37]. The multi‐dimensional operations are complemented by an operator library, which provides a simple way of constructing operators, for example, the forward operator describing an MRI measurement, while automatically providing its adjoint in the case of a linear operator, or the derivative for non‐linear operators. For most use cases it is sufficient to interact with BART using its command line interface, which consists of a set of high‐level tools operating on md‐arrays. All commands take several individual options and typically input and output file names. A single driver command, bart, acts as the unified entry point to BARTs command line interface. It can be called directly from a command line or from scripting languages such as BASH, Python or Matlab. The bart call is followed by a set of general options and the name of the tool which should be run. There are basic tools such as fft which calculates the Fourier transform of an md‐array, as well as many commands for MRI‐specific tasks such as whitening (whiten), coil compression (cc), calibration of sensitivities (ecalib, ncalib), parallel imaging and compressed sensing (pics), model‐based reconstruction (moba), machine‐learning reconstruction (reconet, nlinvnet), and much more. The tools act as building blocks and can be combined in a modular way to construct advanced reconstruction pipelines for all kinds of MRI applications. A complete reconstruction implemented with BART thus typically consists of a script that calls various BART commands that operate on md‐arrays. For real‐time reconstruction, it must be possible to split up, that is, slice, the multi‐dimensional input space and process each subset of the data as it becomes available. In other words, the reconstruction script has to be organized as a pipeline. To support this in BART while preserving its modular character, the tools were enhanced to support streaming of slices of md‐arrays. Slicing Suppose for n∈ℕ that we want to split up an n‐dimensional array X∈ℂN0×N1×…×Nn−1. In the following, the entries Xi0,…,in−1 of X are referenced using the multi‐index i0,…,in−1,ij∈ℕ0andij stream.txt bart -l4 -r - fft -i 3 - out < stream.txt bart -l4 -r x copy x -> stream.txt bart -l4 -r - fft -i 3 - out < stream.txt In a second experiment, we measure the speed‐up for a simple reconstruction pipeline where streaming and looping enable parallelization of consecutive reconstruction steps. A customizable delay is added to the input stream, simulating per‐slice acquisition time. The output of the copy‐tool is then forwarded to the fft‐tool directly via a pipe, causing the processes to run in parallel: bart -l4 -r x copy --delay x - |\ bart -l4 -r - fft -i 3 - out bart -l4 -r x copy --delay x - |\ bart -l4 -r - fft -i 3 - out This is compared to the static variant: bart -l4 -r x copy --delay x - |\ bart fft -i 3 - out where the reconstruction can only begin after the copy‐process has processed all slices. The experiment is repeated for different delays between 0 and 130 ms. Wall‐clock run time is measured using the date program from GNU Coreutils. The experiments were repeated 25 times to measure standard deviation across runs, and performed on a standard workstation using an Intel Xeon W‐2123 CPU running at 3.60 GHz. bart -l4 -r x copy --delay x - |\ bart fft -i 3 - out To assess latency for real‐time MRI, phantom measurements were performed. We use the real‐time reconstruction script (Section 2.6, set to good), and additionally run the experiment without median filter to assess the impact of this post‐processing step on overall latency. Apart from acquisition and the reconstruction, several other factors such as network transfer times, filters used in post processing, and image display are expected to contribute to the total latency. Therefore, we determined several different latencies:Latency in BART is calculated as time difference between arrival of the last spoke and transmission of the reconstructed frame. Preprocessing, reconstruction and post processing were also measured individually. Timestamps are recorded using the gettimeofday function from glibc.Latency BART + Network as seen from the software running on the scanner is measured by taking timestamps when the last spoke is sent over the network and after the corresponding frame has been received. This also includes data transfer times and network delays.End‐to‐end latency, that is, the time between an event happening in the scanner until it is shown on the MRI screen, is measured using an experimental setup as described in [48], correlating movements in a phantom with movement on the screen using a video recording. Additional information is available in Data S2. Latency in BART is calculated as time difference between arrival of the last spoke and transmission of the reconstructed frame. Preprocessing, reconstruction and post processing were also measured individually. Timestamps are recorded using the gettimeofday function from glibc. Latency in BART is calculated as time difference between arrival of the last spoke and transmission of the reconstructed frame. Preprocessing, reconstruction and post processing were also measured individually. Timestamps are recorded using the gettimeofday function from glibc. Latency BART + Network as seen from the software running on the scanner is measured by taking timestamps when the last spoke is sent over the network and after the corresponding frame has been received. This also includes data transfer times and network delays. Latency BART + Network as seen from the software running on the scanner is measured by taking timestamps when the last spoke is sent over the network and after the corresponding frame has been received. This also includes data transfer times and network delays. End‐to‐end latency, that is, the time between an event happening in the scanner until it is shown on the MRI screen, is measured using an experimental setup as described in [48], correlating movements in a phantom with movement on the screen using a video recording. Additional information is available in Data S2. End‐to‐end latency, that is, the time between an event happening in the scanner until it is shown on the MRI screen, is measured using an experimental setup as described in [48], correlating movements in a phantom with movement on the screen using a video recording. Additional information is available in Data S2. Because the MRI acquisition starts as soon as possible without waiting for the BART reconstruction pipeline to be initialized, the latency decreases towards a steady state, which is reported here as steady state latency. Results Performance Numerical Experiments We first present results on the performance of the two main additions to the BART software, streaming and looping. As described in Section 2.7, looping and streaming can under some conditions create computational overhead. Figure 3A shows the overhead when performing a looped FFT compared to normal FFT. For small arrays with many iterations/slices, there is a large relative overhead. For example, the FFT of a 96 × 48 × 100 array takes 50% or 10(1) ms (mean with standard deviation in parentheses) longer when looping over the last dimension. FFT of a 2048 × 1024 × 100 array takes 2% or 400(150) ms longer. For few slices or large problem sizes, the differences between looping and combined operation are small relative to the variation over repetitions. The performance of BART looping and streaming is shown here by visualizing the runtime of an inverse FFT on arrays of varying size. Part A shows a heat map of additional time required when performing a looped FFT relative to the normal/combined FFT, for different problem sizes. The opacity/lightness is based on an adjusted coefficient of variation (aCV): Fully saturated colors represent a distinct overhead, while light colors indicate that the average run‐to‐run difference is at least as big as the calculated overhead. Part B compares the overhead of streaming relative to looping, similar to part A. Part C visualizes the memory advantage of looped versus combined operation for a matrix size of 512 × 256 with varying number of slices. Memory usage is normalized to the amount required for a single slice, that is, array size 512 × 256 × 1. Part D shows the speed‐up due to reduced waiting times in pipeline operation, that is, when the process producing the data (acquisition) and the reconstruction can be parallelized. In Figure 3B, we repeat this analysis for the streaming protocol, comparing the extra time required when performing looping with a streamed input as opposed to looping alone. The maximum overhead is several times smaller than in the previous analysis. Furthermore, the overhead induced by the stream protocol is smaller than the average run‐to‐run variation in most cases. Also, there is no clear trend towards any greater or smaller overhead depending on the matrix size. Figure 3C, illustrates how, with looping, peak memory usage is independent of the number of slices while it grows linearly with normal that is, combined operation. Furthermore, combined streaming and looping enables parallelization of successive data processing steps, and thus can speed up computation. This is illustrated in Figure 3D. Depending on matrix size and delay (slice acquisition time), total elapsed time was reduced down to a minimum of 56% of the reference. End‐To‐End Latency The latency of our RT‐MRI setup was measured using different methods (Section 2.7). The measurement was repeated three times, with almost identical outcome. One instance of these measurements is shown in Figure 4. The BART‐internal timing measurements reveal the amounts of time required for preprocessing, reconstruction and post processing, which are 3(2) ms, 13(2) ms and 13(1) ms, respectively. Relative to the acquisition time of a single frame, which is 13⋅2.1 ms =27.3 ms, all steps are relatively fast. Without the median filter post processing time is 2(1) ms. Latency measurements for RT‐MRI. Part A visualizes results from three different latency measurement methods described previously. Latency determined within BART is further separated into three different steps: Preprocessing, reconstruction and post processing. The times are shown for two measurements, with and without median filter in the post processing chain. Error bars indicate standard deviation over the course of the time series. An initially higher end‐to‐end latency in the first few seconds is excluded for calculating the steady‐state end‐to‐end latency shown here. Part B on the right highlight details of the end‐to‐end latency measurement. Every colored line corresponds to a movement in the end‐to‐end latency measurement setup. Solid lines are calculated from the optically observed movement, while dashed lines correspond to the movement observed in the reconstructed images shown on the MRI console. The optimally shifted MRI movement curves are shown with transparent dashed lines. In (C), the resulting time series of end‐to‐end latency measurements is shown. The latency measured from the perspective of the software running on the scanner matches the total latency measured within BART with a difference of approximately 5 ms corresponding to the network transfer. The end‐to‐end latency of 205(20) ms is several times larger than the other two latency measurements with only 28(2) ms from the BART reconstruction. This discrepancy between end‐to‐end latency compared to the other measurements has multiple sources. For instance, the acquisition time itself contributes to the end‐to‐end latency. Furthermore, time‐dependent filters such as the median filter do not affect the signal processing time too much, but have a large impact on the end‐to‐end latency. Without median filter, an end‐to‐end latency of 146(13) ms is achieved. Several other possible sources are treated in the discussion. Real‐Time Applications In our experimental setup, the simple reconstruction based on the adjoint NUFFT can be run in real‐time on standard hardware without use of a GPU as demonstrated by the latency graph shown Figure 5E. Iterative reconstruction methods such as RT‐NLINV can be used to achieve high image quality, which can be further improved through post processing, as seen in Figure 5B–D. The initially higher, but then quickly decreasing latency in these cases is probably due to initialization of the GPU and precomputing steps. A comparison between non‐causal reconstruction and real‐time capabable reconstruction can be found in the literature [20]. Videos of the reconstruction are available in Data S3. Five representative frames covering one heartbeat from a real‐time acquisition of a cardiac short‐axis view, reconstructed with different quality settings for the real‐time reconstruction script (Section 2.6). Column A shows images reconstructed with fast, column B good and column C high quality settings. Part D provides an enlarged view of the high quality setting. The graph E shows the numerically determined latency of all configurations. Figure 6 compares static and dynamic coil compression and demonstrates the impact of aligned dynamic coil compression as described in Section 2.5. In Figure 6A, it can be seen that noise is increased after a change of slice position, as the compression matrix is not adapted to the changed coil profiles in the new slice. Dynamic coil compression leads to problems as shown in Figure 6B: The coil profiles show distinct jumps over their time course and this then also affects the reconstructed images. These discontinuities disappear when using aligned dynamic coil compression. For comparison, we also show coil and image profiles from reconstruction with static coil compression, which exhibit no discontinuities/brightness jumps. Part A compares two frames from a RT‐MRI of the heart with different strategies for coil compression, either re‐using the compressing matrix or recomputing and aligning it for every frame. The red arrows highlight abrupt changes in image magnitude which correlate with the discontinuities in the virtual coil profiles. Part B shows a time series from a line through the heart, with the line being shown in the small inset image. The bottom row shows the time series for the first two virtual coil profiles, the top row shows the actual images. On the top left, every frame was independently compressed, on the top right, compression matrices were aligned using geometric coil compression. For comparison, line profiles for coil compression done on the first frame/complete dataset are shown in the bottom left/right. Discussion Streaming functionality was added to the BART reconstruction software as new feature. The implementation achieves high modularity and interoperability for real‐time reconstruction pipelines by relying on standard operating system utilities. Based on a set of new options, all tools included in the toolbox can be used in real‐time reconstruction pipelines. The streaming extension was tested and benchmarked in numerical experiments and using phantom and in vivo measurements. As it is now possible to parallelize subsequent reconstruction steps in a pipeline, a speed‐up can be achieved for conventional reconstruction pipelines simply by chaining BART commands. Under certain circumstances, the overhead associated with streaming and looping can have a negative effect on performance, but in all tested conditions overhead was either small in absolute units, small compared to the total computation time, or small relative to the run‐to‐run variations. In most practical applications, the overhead should not be relevant or even be unnoticeable. Naturally, there is some overlap between BART and other reconstruction frameworks, such as the Gadgetron [31] and also with vendor pipelines. Nevertheless, BART is not meant to replace such other frameworks, but to provide a set of versatile tools that can be used to solve specific problems in a variety of scenarios. In fact, it is very common to only use specific BART tools as part of other projects and this is facilitated by BART's open and modular design. So far, this was exploited mostly for offline reconstruction. By providing a low‐latency path for continuous data exchange, BART can now also be integrated into real‐time processing pipelines. To demonstrate the flexibility of the new streaming framework, we showed examples that extend from simple NUFFT‐based reconstruction which can be used even on desktop systems to advanced iterative reconstruction that achieves image quality and makes use of GPU acceleration. The advantages of dynamic coil compression was shown, demonstrating how advanced functionality can be implemented in a modular pipeline. We provide a versatile reconstruction script for real‐time as part of the BART toolbox that includes all the functionality described in this work. New BART functionality such as removal of phase pole artifacts [49] were also already integrated and shown to be beneficial in preliminary experiments. We furthermore measured the latency of our interactive RT‐MRI setup for a full reconstruction pipeline. Here, we found that the part of the latency corresponding to the BART pipeline is only a small fraction of the overall end‐to‐end latency. Processing time in BART, including network transfer, is below 30 ms, which is acceptable even for demanding application such as cardiac interventional MRI. End‐to‐End latency is considerably higher with 200 ms, which is in part explained by the response of the median filter used in post processing, and the duration of the data acquisition itself. As shown previously [50] the temporal regularization does not add substantial latency. Another source of end‐to‐end latency is image processing in the vendor software, which includes for example, distortion correction and displaying of the images. For integration with the MRI scanner, we used a custom, vendor‐specific software for data import/export. However, we also already performed initial experiments using scanner integration based on MRD streams, using the BART ismrmrd tool to interface at the start/end of the reconstruction pipeline. With vendors increasingly providing open interfaces, this should simplify the integration of BART into custom pipelines. Conclusion The streaming extension to the BART framework enables efficient reconstruction for RT‐MRI applications with low latency, while retaining the full modularity of the BART framework. Funding This work was funded by DZHK (German Centre for Cardiovascular Research) grant 81X4300119 and in part by NIH under grant U24EB029240. Conflicts of Interest The authors declare no conflicts of interest. Supporting information Data S1: Real‐time MRI reconstruction pipeline graph. Figure S1: State‐of‐the‐art reconstruction pipeline for real‐time MRI. Different sections of the reconstruction pipeline are highlighted, and characteristic parts are also shown in a zoomed‐in, abbreviated version next to the full graph. Round nodes represent BART invocations while edges between tool nodes represent pipes. The invocation of the real‐time reconstruction script corresponding to this graph is shown at the bottom, along with an explanation of command line options. The figure is based on an auto‐generated graph, which was created by tracing every BART tool invocation and translating this trace into a graph description language, and was rendered with dot. Data S2: End‐to‐end latency measurement. Figure S2: Part A: Schematic overview of the end‐to‐end latency measurement experiment setup. A water‐filled test‐tube (blue) inside the MRI scanner is pulled up from the control room using a long string (yellow). Images from the real‐time sequence are shown on the MRI console screen. A smartphone camera records movement on the MRI screen and movement of a paper tag (gray) attached to the string. A single frame from the resulting video is shown on the top right. Part B illustrates how the resulting video is processed: The video is segmented, and the obtained object velocity curves are cross‐correlated to obtain the delay between both movements. Data S3: Real‐time MRI reconstruction videos. Corresponding video for Figure 5, covering three heartbeats, starting at the same time as the timeseries shown in Figure 5. The ordering matches the original figure, that is, images were reconstructed using (from left to right) the fast, good, and high‐quality settings of the rtreco.sh reconstruction script.