Semantic Mosaicing of Histo-Pathology Image Fragments using Visual Foundation Models
Stefan Brandstätter [email protected], Maximilan Köller [email protected], Philipp Seeböck [email protected], Alissa Blessing [email protected], Felicitas Oberndorfer [email protected], Svitlana Pochepnia svitl · 2025
Read Semantic Mosaicing of Histo-Pathology Image Fragments using Visual Foundation Models by Stefan Brandstätter [email protected], Maximilan Köller [email protected], Philipp Seeböck [email protected], Alissa Blessing [email protected], Felicitas Oberndorfer [email protected], Svitlana Pochepnia svitl (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03524.txt.
Introduction
Stefan Brandstätter and co-authors published this study in 2025. It stitches tissue sample fragments into whole slide images by meaning, not just shape. Pathology samples often exceed one slide, so fragments must be rejoined to study whole tumors. Older stitchers match fragment edges, and fail when tissue is lost, warped, or unevenly stained. The team's SemanticStitcher reads deep features from a vision foundation model trained on tissue images.
Those features flag which areas in different fragments were once neighbors. From many candidate matches, a robust fit works out each fragment's pose. The mosaic then forms a single whole mount slide. On three data sets, the method beat the best edge-matching tools on correct joins. Reliable mosaics, the authors note, are a gate to scaling tissue analysis with computers.
Catalog record
| CREATOR | Stefan Brandstätter [email protected], Maximilan Köller [email protected], Philipp Seeböck [email protected], Alissa Blessing [email protected], Felicitas Oberndorfer [email protected], Svitlana Pochepnia svitl |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-03524
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Semantic Mosaicing of Histo-Pathology Image Fragments using Visual Foundation Models Stefan Brandstätter [email protected], Maximilan Köller [email protected], Philipp Seeböck [email protected], Alissa Blessing [email protected], Felicitas Oberndorfer [email protected], Svitlana Pochepnia [email protected], Helmut Prosch [email protected], Georg Langs [email protected] Abstract In histopathology, tissue samples are often larger than a standard microscope slide, making stitching of multiple fragments necessary to process entire structures such as tumors. Automated stitching is a prerequisite for scaling analysis, but is challenging due to possible tissue loss during preparation, inhomogeneous morphological distortion, staining inconsistencies, missing regions due to misalignment on the slide, or frayed tissue edges. This limits state-of-the-art stitching methods using boundary shape matching algorithms to reconstruct artificial whole mount slides (WMS). Here, we introduce SemanticStitcher using latent feature representations derived from a visual histopathology foundation model to identify neighboring areas in different fragments. Robust pose estimation based on a large number of semantic matching candidates derives a mosaic of multiple fragments to form the WMS. Experiments on three different histopathology datasets demonstrate that SemanticStitcher yields robust WMS mosaicing and consistently outperforms the state of the art in correct boundary matches. † † shortheadings: SemanticStitcher / SemanticStitcher † † firstpage: 1 keywords Whole-Mount Sectioning (WMS), UNI, Histopathology, Image Stitching, Foundation Model 1 Introduction While microscope slides are essential for pathology, their size limits full specimen analysis. ( Duan et al. 2024 ) Whole-Mount histopathology addresses this but introduces new scanning challenges. Artificial WMS mosaicing offers a solution through fragment alignment. Whole-Mount histopathology (WMH) is a comprehensive technique examining the entire cross-section of a specimen resulting in a Whole-Mount Sectioning (WMS) large-format slide. ( Cimadamore et al. 2020 ) It captures the full spatial distribution and morphological features of tissue relevant for diagnosis and research. WMS enhances histopathology-imaging correlation, reduces cutting artifacts, and preserves tissue context. ( Schouten et al. 2024 ) However, it also introduces challenges, including the need for larger, more costly scanners, and technical limitations in capturing oversized slides. ( Duan et al. 2024 ) To address these limitations, ongoing research focuses on creating artificial WMS images by aligning tissue fragments, to obtain advantages of WMS while maintaining standardized image acquisition protocols. ( Chappelow et al. 2011 ; Penzias et al. 2016 ; Schouten et al. 2024 ) Related work Several stitching methods have been proposed. HistoStitcher ( Chappelow et al. 2011 ) requires manual landmark selection and transformation tuning (e.g., reflection, scaling), making it too labor-intensive for clinical use AutoStitcher ( Penzias et al. 2016 ) was the first fully automated method, using a domain-specific cost function based on L2-norm histogram differences and a misalignment term, but still depends on manual fragment labels. PythoStitcher ( Schouten et al. 2024 ) is the current state-of-the-art, applying a boundary-based, multi-resolution strategy for high-resolution mosaicing without manual input or extra cost functions. However, it performs poorly on irregularly shaped or equally sized boundary fragments. To address these limitations while retaining WMS benefits, we propose a novel automatic mosaicing method To take advantage of WMS while addressing associated challenges, we propose a novel method for automatic mosaicing of artificial WMS from given tissue fragments. Contribution We introduce SemanticStitcher, an automated WMS mosaicing method that aligns arbitrary boundary fragments using direct image content stitching. Unlike state-of-the-art boundary-based algorithms, it requires no prior knowledge of tissue shape, arrangement, or fragment count. Instead, it leverages semantic features extracted with the help of a foundation model to effectively compare the content of image patches at high resolution, enabling robust and accurate alignment of fragment boundaries. 2 Method We propose SemanticStitcher , a method for accurately aligning digitized tissue fragments \mathbf{X}\in\mathbb{R}^{h\times w} to reconstruct an artificial WMS (Fig. 1 ). The algorithm selects consecutive fragments from a fragment pool, pairs it with its best-matching counterpart, and computes the rotation matrix \mathbf{R} and the translation vector \mathbf{t} required for precise alignment of the fragment. The approach consists of two stages: (1) Identifying neighboring fragment pairs and corresponding semantic match candidates and (2) robustly estimating their spatial alignment for mosaicing. Stage 1: Fragment pairing As a simple preprocessing step, for each fragment \mathbf{X} in the fragment pool we remove the background (OTSU ( Otsu 1979 ) ) and detect the boundary \mathcal{B}_{X} ( Suzuki and be 1985 ) . Along the fragment boundaries, patches \smash{\mathbf{P}_{X}^{(k)}\in\mathbb{R}^{ph\times pw}} are sampled at fixed intervals. Each patch is encoded into a feature vector \smash{\mathbf{S}:\mathbf{P}_{X}^{(k)}\mapsto\mathbf{f}_{X}^{(k)}\in\mathbb{R}^{K}} using a pre-trained semantic encoding model. In our experiments we evaluate the UNI and CONCH foundation model ( Chen et al. 2024 ; Lu et al. 2023 ) . A random fragment \mathbf{X}_{M} out of the pool is selected as the moving fragment. All other fragments are treated as fixed fragments \mathbf{X}_{F} . To determine the best fragment match, we compute the cosine similarity between all feature vectors of the moving fragment \mathbf{X}_{M} and all boundary patch feature vectors of all fixed fragments \mathbf{X}_{F} . For each feature vector in the moving fragment \smash{\mathbf{f}_{X_{M}}^{(k)}} , we identify the most similar feature vector on the fixed fragments \smash{\mathbf{f}_{X_{F}}^{(j)}} , forming candidate matches. To select a matching fragment, we sum up the cosine similarity for each \mathbf{X}_{F} and select the fixed fragment with the highest score as the optimal match for \mathbf{X}_{M} . Stage 2: Fragment alignment To align the fragments, we first apply fragment informed filtering, where only the feature vectors of the matched fragment pair (\mathbf{X}_{M},\mathbf{X}_{F}) from Stage 1 are retained for alignment. We then identify candidate matches between \mathbf{X}_{M} and \mathbf{X}_{F} by finding feature pairs with the highest similarity. The candidate matches form the basis for a robust estimation of alignment parameters between the fragments using RANSAC ( Fischler and Bolles 1981 ) . It mitigates the impact of outliers and incorrectly matched pairs, and identifies the pose parameters consistent with a majority of matches. Once \mathbf{X}_{M} is aligned with \mathbf{X}_{F} , the resulting composite image replaces the two initial fragments in the fragment pool, and the fragment selection and alignment is repeated. This iterative process continues until a single fragment remains, forming the complete WMS. In the following we provide details of the individual steps of the approach. Figure 1: SemanticStitcher: In Stage 1, a fragment is randomly selected from the fragment pool and paired with its most compatible counterpart. In Stage 2, the optimal transformation aligning the paired fragments is computed. Patch extraction For patch extraction, in our experiments, we begin with a boundary point and identify the next boundary point at a distance of 224 pixels, corresponding to the input size of the UNI encoder. The patch \smash{\mathbf{P}_{X}^{(k)}} is then extracted perpendicular to the line connecting these two points. To ensure comprehensive coverage of the boundary \mathcal{B}_{X} , patches are extracted with an overlap of half the patch size. Patch extraction is shifted 10 pixels inward to avoid black pixels from frayed edges. This process is performed along the entire extracted boundary. Patch encodings To encode the patches \smash{\mathbf{P}_{X}^{(k)}} into meaningful feature vector representations \smash{\mathbf{f}_{X}^{(k)}} , we utilize the state-of-the-art foundation model UNI, a general-purpose, self-supervised model for pathology. ( Chen et al. 2024 ) We follow the preprocessing described in the UNI paper. Feature vector matching To establish correspondences between the moving fragment \mathbf{X}_{M} and fixed fragment \mathbf{X}_{F} , we consider both local