Beyond Imaging: Vision Transformer Digital Twin Surrogates for 3D+T Biological Tissue Dynamics
Kaan Berke Ugurlar, Joaquín de Navascués, Michael Taynnan Barros · 2025
Read Beyond Imaging: Vision Transformer Digital Twin Surrogates for 3D+T Biological Tissue Dynamics by Kaan Berke Ugurlar, Joaquín de Navascués, Michael Taynnan Barros (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15883.txt.
Introduction
Ugurlar, de Navascués, and Barros present a model, published in 2025, that acts as a digital twin for living tissue. Watching tissues change over time in three dimensions produces rich imaging data. Making sense of it, and predicting what happens next, is hard. Their network learns from time-lapse images of a fruit fly's gut. It builds on vision transformers first trained to find structure in unlabeled images, a method that teaches itself from data.
A multi-view fusion step combines images taken at different depths. The trained model can then replay the gut's dynamics with high fidelity, keeping shapes and fine features intact. It can also forecast how the tissue will evolve. Such surrogates let biologists test ideas in simulation before running wet experiments. The work points toward predictive models of tissue health and disease built straight from imaging.
Catalog record
| CREATOR | Kaan Berke Ugurlar, Joaquín de Navascués, Michael Taynnan Barros |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-15883
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Imaging: Vision Transformer Digital Twin Surrogates for 3D+T Biological Tissue Dynamics Kaan Berke Ugurlar, Joaquín de Navascués, Michael Taynnan Barros Abstract Understanding the dynamic organization and homeostasis of living tissues requires high-resolution, time-resolved imaging coupled with methods capable of extracting interpretable, predictive insights from complex datasets. Here, we present the Vision Transformer Digital Twin Surrogate Network (VT-DTSN), a deep learning framework for predictive modeling of 3D+T imaging data from a biological tissue. By leveraging Vision Transformers pretrained with DINO (Self-Distillation with NO Labels) and employing a multi-view fusion strategy, the VT-DTSN learns to reconstruct high-fidelity, time-resolved dynamics of a Drosophila midgut tissue while preserving morphological and feature-level integrity across imaging depths. The model is trained with a composite loss prioritizing pixel-level accuracy, perceptual structure, and feature-space alignment, ensuring biologically meaningful outputs suitable for in silico experimentation and hypothesis testing. Evaluation across layers and biological replicates demonstrates the VT-DTSN’s robustness and consistency, achieving low error rates and high structural similarity while maintaining efficient inference capability through model optimization. This work establishes VT-DTSN as a feasible, high-fidelity surrogate for cross-timepoint reconstruction, for studying tissue dynamics, enabling computational exploration of cellular behaviors and homeostasis to complement time-resolved imaging studies in biological research. Index Terms— 3D+T imaging; confocal microscopy; digital twin; surrogate modeling; vision transformer; epithelial homeostasis; Drosophila midgut 1 INTRODUCTION High‑resolution 3D+T confocal imaging of the Drosophila midgut enables quantitative study of epithelial homeostasis and regeneration. However, exploiting its depth‑varying signal and heterogeneous cellular architecture remains computationally challenging [ 1 , 2 , 3 ] . The sheer complexity and volume of this data present significant analytical challenges. A typical acquisition comprises several optical sections per Z‑stack at high resolution with micrometric voxel size, repeated across independent biological replicates and time points. Depth‑dependent scattering and photobleaching reduce SNR in deeper layers, while phototoxicity limits repeated imaging of the same specimen. These constraints create partially observed, noisy 3D+T volumes where long‑range spatial dependencies (across layers) and temporal consistency must be recovered from limited sampling data. Traditional computational methods fall short in capturing the nuanced spatial-temporal patterns inherent in tissue dynamics, limiting our ability to fully leverage this wealth of information. Standard convolutional neural networks (CNNs) pipelines emphasize local receptive fields and under‑represent long‑range cross‑layer structure [ 4 , 5 ] . recurrent neural networks (RNNs) and temporal CNNs mitigate this only partially and are costly to scale on volumetric data [ 6 ] . Moreover, models trained on narrowly curated imaging conditions often fail to generalize across biological replicates with variable contrast, labeling, and depth attenuation. The specialized nature of experimental tissue imaging datasets leads to difficulties in model generalization. Furthermore, the sophisticated spatial-temporal behaviour of tissue processes demands more than what standard CNN and RNN architectures can offer. What is currently missing is a data‑driven surrogate that can reconstruct and predict midgut dynamics from sparse or noisy 3D+T stacks with fidelity to morphology and features, while remaining efficient enough for near real‑time use. Figure 1: Conceptual schematic illustrating the Vision Transformer Digital Twin Surrogate Network (VT-DTSN) workflow for Drosophila midgut homeostasis studies. High-resolution 3D+T imaging datasets of the midgut are acquired and input into the VT-DTSN pipeline, where Vision Transformer-based feature extraction and fusion reconstruct dynamic tissue states across time and depth. To address these limitations, we pursue a digital‑twin surrogate: a high‑fidelity, data‑driven model that reproduces observed tissue dynamics for in silico experimentation. Here, ‘ digital twin ’ denotes a statistical surrogate rather than a mechanistic simulator with explicit biophysical laws. By learning from high-dimensional, time-resolved imaging data, such models can emulate the behaviour of living tissues in silico, enabling predictive simulation, hypothesis testing, and closed-loop experimental design without the constraints of physical experimentation alone. This enables integration with diverse biological datasets without requiring explicit mechanistic parameterization, making the approach adaptable to other tissues and imaging modalities. In this context, we propose the Vision Transformer Digital Twin Surrogate Network (VT-DTSN), a framework that uses deep learning not merely for image classification or segmentation, but to generate predictive, high-fidelity reconstructions of dynamic tissue behaviours, effectively serving as a computational proxy for live biological systems. Vision Transformers, via self‑attention, model long‑range spatial relations across Z‑slices more naturally than convolutional hierarchies. A digital twin surrogate model of the midgut enables predictive, in silico experimentation that complements time-resolved imaging, allowing researchers to simulate tissue responses to genetic, pharmacological, or mechanical perturbations before committing to labor-intensive experimental procedures. By capturing the layered epithelial architecture and cell-type-specific dynamics across time and depth, VT-DTSN allows high-fidelity reconstruction and cross-timepoint reconstruction/prediction across replicates of midgut dynamics, offering insights into tissue organization and cellular behaviors that are impractical to measure continuously in vivo. This capability opens new pathways understanding how cellular heterogeneity and spatial structure influence gut homeostasis in Drosophila . Our approach harnesses the power of Vision Transformers (ViTs), utilizing their self-attention mechanisms to capture visual patterns crucial for understanding cell dynamics. We innovate further by integrating DINO ( Self‑Distillation with No Labels ) pretraining, which enhances the ViTs’ ability to assimilate spatial context and temporal cues crucial in cellular imaging. This methodology is complemented by a multi-view fusion strategy, augmenting the model’s capability to synthesize diverse perspectives into a cohesive understanding of cellular behaviour. Self‑supervised DINO pretraining provides robust features under intensity shifts and staining variability typical of confocal imaging. A multi‑view fusion of ViT branches encourages consistency across lateral and depth cues, improving reconstructions in low‑SNR layers. Our custom training process and loss formulation are designed to prioritize not just pixel accuracy, but also perceptual similarity and biological fidelity, which are essential for meaningful interpretations in biological research and effective surrogate modeling. Rigorous evaluation using metrics such as Mean Squared Error (MSE), Structural Similarity Index (SSIM), and Cosine Similarity ensures that the VT-DTSN aligns closely with authentic biological patterns observed in Drosophila midgut dynamics. Furthermore, to meet the demands of real-time analysis in time-resolved imgaging experimental workflows, we implement model optimization strategies including pruning and mixed-precision inference, resulting in a computationally efficient and high-fidelity surrogate model. Recent progress in self‑supervised ViT pretraining and mixed‑precision inference makes it feasible to train robust volumetric surrogates and to deploy them at interactive speeds in imaging workflows. The key contributions of this work are: • Development of the VT-DTSN, a ViT-based surrogate digital twin model tailored for predictive reconstruction of dynamic tissue imaging data. This provides a computational proxy that can emulate midgut tissue behavior from limited imaging data, reducing the need for exhaustive physical experiments. • Implementation of DINO-based ViT pretraining to enhance feature representation aligned with the complexities of biological imagery. This improves robustness to depth-dependent signal loss and variability in labeling intensity, allowing the model to generalize across biological replicates. • Introduction of a multi-view ViT fusion strategy to enrich spatiotemporal feature integration and predictive accuracy. This ensures that morphological features are consistently reconstructed across Z-stack layers, even in low-SNR regions, preserving biologically relevant structures. • A custom loss formulation emphasizing biological fidelity, perceptual structure, and pixel-wise precision. This alignment between training objectives and evaluation criteria ensures reconstructions are both numerically accurate and interpretable for downstream biological analysis. • Optimization of the VT-DTSN for real-time predictive analysis within experimental pipelines. This enables near-instantaneous feedback during live imaging sessions,