Patch2Loc: Learning to Localize Patches for Unsupervised Brain Lesion Detection

Hassan Baker, Austin J. Brockmeier · 2025

Read Patch2Loc: Learning to Localize Patches for Unsupervised Brain Lesion Detection by Hassan Baker, Austin J. Brockmeier (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-22504.txt.

Introduction

Hassan Baker and Austin Brockmeier teach a neural network the layout of the brain so it can flag lesions, in a 2025 paper. Radiologists hunt MRI scans for abnormal patches such as tumors, and computer aids can help. Most aids need many hand-labeled examples of lesions, which are slow and costly to make.

Their method, Patch2Loc, trains only on healthy scans. The network learns to name where a small patch came from inside a brain slice. At test time, patches from abnormal tissue confuse it, giving high error and spread in its guesses. Those trouble spots form a heat map that highlights likely lesions. Learning from healthy tissue alone could bring lesion detection to hospitals that lack labeled data.

Catalog record

CREATORHassan Baker, Austin J. Brockmeier
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-22504

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Patch2Loc: Learning to Localize Patches for Unsupervised Brain Lesion Detection Hassan Baker, Austin J. Brockmeier Abstract Detecting brain lesions as abnormalities observed in magnetic resonance imaging (MRI) is essential for diagnosis and treatment. In the search of abnormalities, such as tumors and malformations, radiologists may benefit from computer-aided diagnostics that use computer vision systems trained with machine learning to segment normal tissue from abnormal brain tissue. While supervised learning methods require annotated lesions, we propose a new unsupervised approach (Patch2Loc) that learns from normal patches taken from structural MRI. We train a neural network model to map a patch back to its spatial location within a slice of the brain volume. During inference, abnormal patches are detected by the relatively higher error and/or variance of the location prediction. This generates a heatmap that can be integrated into pixel-wise methods to achieve finer-grained segmentation. We demonstrate the ability of our model to segment abnormal brain tissues by applying our approach to the detection of tumor tissues in MRI on T2-weighted images from BraTS2021 and MSLUB datasets and T1-weighted images from ATLAS and WMH datasets. We show that it outperforms the state-of-the art in unsupervised segmentation. The codebase for this work can be found on our GitHub page . Fig. 1: Schematic diagram for Patch2Loc. (Top) Training stage: A randomly selected patch from a normal MRI slice (patches with less than 20\% of its content as brain tissues, as in the yellow patch, are rejected) is paired with its two-dimensional location Y and slice index A . The relationship between the image patch, slice, and its location is modeled as a conditional distribution P_{\theta}(Y|X,A) using a 2D Gaussian distribution defined by the mean the variance of each coordinate that are functions of the patch and slice index. (Bottom) Inference stage: Overlapping patches with a fixed stride are extracted as in convolution from an MRI slice and the model is applied to each patch. The norm of the difference between the model’s predicted mean and the patch’s true location creates an error map. Likewise, the sum of the variances create a variance map. Together the norm of the error and variance can identify anomalous patches. I Introduction Detecting and localizing abnormal brain tissue through non-invasive neuroimaging is a critical medical task with profound implications for patients’ well-being and overall quality of life. Early detection and intervention can mitigate severe consequences in cases such as incipient brain tumors or cortical malformations, which are associated with drug-resistant pediatric epilepsy [ 1 ] . The intricate structural complexity of the brain necessitates the expertise of neurologists and radiologists for accurate diagnosis. The development of computer-aided diagnostics at an expert level is motivated by the shortage of neuroradiologists [ 2 ] . Magnetic resonance imaging (MRI) has emerged as a reliable modality for detecting lesions within the brain [ 3 , 4 , 5 ] . MRI offers the distinct advantage of being radiation-free and non-invasive and different sequences can produce various tissue contrasts. In recent years, significant strides have been made in leveraging machine learning, particularly deep learning techniques for computer vision [ 6 ] to automate the detection and localization of abnormal brain tissues in neuroimages. Methods generate abnormality scores at the pixel or voxel-level that can be thresholded to segment images abnormal and normal regions. This progress is largely attributable to the availability of extensive datasets and advanced deep learning methodologies. While models trained with supervision may be expected to exhibit superior performance, they require extensive annotations delineating the anomalies. However, the release of detailed annotations is limited, especially for rare abnormalities [ 7 , 8 ] . This motivates the improvement of unsupervised learning methods that use only putative normal images that can then be applied for abnormal brain tissue detection. Possibly using a minimal set of labeled images for hyper-parameter selection. As discussed in Section V , state-of-the-art methods use autoencoders, denoising autoencoders, and diffusion models learn to reconstruct or denoise normal images based on the typical structure, such that the reconstructions of abnormal portions of images is closer to typical structure and the reconstruction errors highlight abnormalities. In particular, denoising diffusion probabilistic models (DDPMs) have recently outperformed other generative and restoration-based approaches for unsupervised abnormality detection [ 9 , 10 , 11 , 12 , 13 ] . The key idea is to train a DDPM on normal brain MRI volumes (3D images) or slices (2D image taken from the 3D image of the brain). Since the model learns the patterns of normal brains, if it is fed with slices contain abnormal regions, it inpaints abnormal regions with normal patterns. The reconstruction error, typically measured as the pixel-wise difference between the original and reconstructed images, serves as a heatmap for potential anomalies, with higher errors indicating regions more likely to contain abnormalities. Despite their effectiveness, DDPMs suffer from the noise paradox [ 9 , 14 ] , where the level of added noise during inference significantly impacts performance: higher noise levels remove more anomalies but increase false positives due to over-inpainting of normal tissues. This trade-off complicates the detection process and limits DDPM’s reliability in clinical settings. Furthermore, DDPMs and the vast majority of existing unsupervised abnormality segmentation methods for brain MRI rely on processing entire slices. Thus, these methods detect localized lesions from a “global” context. In contrast, analyzing local patterns in isolation, given their location, may provide more focused detection of abnormalities. With this motivation, we introduce Patch2Loc , a novel framework tailored for achieving unsupervised abnormality segmentation in neuroimaging. We create a self-supervised task, predicting the location of a patch, such that given the true location the differences and uncertainty with the prediction can identify local anomalies. I-A Main Contributions Our approach offers several key advantages: 1. Spatially-Aware Local Feature Learning: Patch2Loc learns to predict the spatial origin of a given patch within the brain. By encoding local anatomical and tissue structure patterns from normal tissue patches, the model can effectively distinguish abnormal patches based on deviations in location prediction. 2. Uncertainty-Guided Anomaly Detection: The variance in Patch2Loc’s predictions serves as an uncertainty estimate, enabling more robust abnormality detection. By combining location error and uncertainty, we derive an enhanced abnormality score that improves detection performance. 3. Minimal Hyperparameter Dependence: Unlike DDPM-based methods, which require hyperparameter tuning (e.g., noise level selection), Patch2Loc is self-supervised and adaptable. The most anomalous patches can be found by the abnormality score. By shifting the focus to local features and leveraging self-supervised learning, Patch2Loc’s abnormality score is intuitive and easy to interpret and avoidance of extensive hyperparameter tuning and labeled data increases clinical applicability. II Methodology The idea underlying Patch2Loc is that there is a strong relationship between patch content and location, due to the local patterns in structural neuroimages, that is used by neuroradiologists when identifying structural abnormalities. With sufficient sampling across the population, this relationship can be modeled using machine learning. As each brain has slightly different sizes, the images are first registered using rigid transformation. Fig. 1 shows a summary schematic diagram for methodology behind Patch2Loc. II-A Problem Formulation Let (Y_{1},Y_{2},A)\in[0,100]^{3} denote the 3D location of a rectangular image patch X\in\mathcal{X}\subset\mathbb{R}_{\geq 0}^{S_{1}\times S_{2}} within the brain taken at the 2D location

Read the complete text