From Redaction to Restoration: Deep Learning for Medical Image Anonymization and Reconstruction

Adrienne Kline, Abhijit Gaonkar, Daniel Pittman, Chris Kuehn, Nils Forkert, Center for Artificial Intelligence, BCVI, Northwestern Medicine, Chicago, IL, USA Department of Electrical and Computer Engineering, Northwestern University, Chicago, IL, USA Department of Surgery, Northwestern University, Chicago, IL, USA Xtasis Inc., Chicago, IL, USA Medtronic, USA Departments of Radiology and Clinical Neurosciences, University of Calgary, Calgary, AB, Canada. Corresponding author: [email protected] · 2026

Read From Redaction to Restoration: Deep Learning for Medical Image Anonymization and Reconstruction by Adrienne Kline, Abhijit Gaonkar, Daniel Pittman, Chris Kuehn, Nils Forkert, Center for Artificial Intelligence, BCVI, Northwestern Medicine, Chicago, IL, USA Department of Electrical and Computer Engineering, Northwestern University, Chicago, IL, USA Department of Surgery, Northwestern University, Chicago, IL, USA Xtasis Inc., Chicago, IL, USA Medtronic, USA Departments of Radiology and Clinical Neurosciences, University of Calgary, Calgary, AB, Canada. Corresponding author: [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11376.txt.

Introduction

Kline and four colleagues anonymize medical images without wrecking downstream analysis. Standard de-identification often strips useful non-identifying content. Their end-to-end pipeline first detects and redacts regions likely to hold protected health information, such as burned-in text and metadata, then inpaints those areas with plausible anatomy via a latent-diffusion model. Tests on privacy, image quality, and task metrics show visually coherent de-identified volumes that keep fidelity for downstream models while substantially cutting re-identification risk, easing large-scale sharing.

Catalog record

CREATORAdrienne Kline, Abhijit Gaonkar, Daniel Pittman, Chris Kuehn, Nils Forkert, Center for Artificial Intelligence, BCVI, Northwestern Medicine, Chicago, IL, USA Department of Electrical and Computer Engineering, Northwestern University, Chicago, IL, USA Department of Surgery, Northwestern University, Chicago, IL, USA Xtasis Inc., Chicago, IL, USA Medtronic, USA Departments of Radiology and Clinical Neurosciences, University of Calgary, Calgary, AB, Canada. Corresponding author: [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53054
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.11376

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Redaction to Restoration: Deep Learning for Medical Image Anonymization and Reconstruction Adrienne Kline, Abhijit Gaonkar, Daniel Pittman, Chris Kuehn, Nils Forkert, Center for Artificial Intelligence, BCVI, Northwestern Medicine, Chicago, IL, USA Department of Electrical and Computer Engineering, Northwestern University, Chicago, IL, USA Department of Surgery, Northwestern University, Chicago, IL, USA Xtasis Inc., Chicago, IL, USA Medtronic, USA Departments of Radiology and Clinical Neurosciences, University of Calgary, Calgary, AB, Canada. Corresponding author: [email protected] Abstract Removing patient-specific information from medical images is crucial to enable sharing and open science without compromising patient identities. However, many methods currently used for deidentification have negative effects on downstream image analysis tasks because of removal of relevant but non-identifiable information. This work presents an end-to-end deep learning framework for transforming raw clinical image volumes into de-identified, analysis-ready datasets without compromising downstream utility. The methodology developed and tested in this work first detects and redacts regions likely to contain protected health information (PHI), such as burned-in text and metadata, and then uses a generative deep learning model to inpaint the redacted areas with anatomically and imaging plausible content. The proposed pipeline leverages a lightweight hybrid architecture, combining CRNN-based redaction with a latent-diffusion inpainting restoration module (Stable Diffusion 2). We evaluate the approach using both privacy-oriented metrics, which quantify residual PHI and success of redaction, and image-quality and task-based metrics, which assess the fidelity of restored volumes for representative deep learning applications. Our results suggest that the proposed method yields de-identified medical images that are visually coherent, maintaining fidelity for downstream models, while substantially reducing the risk of patient re-identification. By automating anonymization and image reconstruction within a single workflow, and dissemination of large-scale medical imaging collections, thereby lowering a key barrier to data sharing and multi-institutional collaboration in medical imaging AI. K eywords medical imaging, de-identification, image restoration, deep learning, generative artificial intelligence 1 Introduction Machine learning models trained using this medical imaging data have considerably contributed to the speed and repeatability of diagnostic tasks, sometimes even outperforming human experts in assessments of chest x-rays, retinal scans, and echocardiography assessments [ 29 ] [ 13 ] [ 20 ] . However, training machine learning models on small or noisy datasets calls limits the reproducibility and generalizability of these models for real-world deployments. Large conglomerate imaging datasets have facilitated open-source repositories of information for the development of generalizable machine learning and artificial intelligence models for computer-aided diagnosis support and clincial decision making [ 2 ] [ 25 ] [ 12 ] . In fact, these databases provide current state-of-the-art benchmarks for the development and testing of novel methodologies [ 4 ] [ 8 ] . Therefore, data cleaning and subsequently sharing are a necessary first step for facilitating large imaging repositories which can impact downstream health inferences drawn from algorithmic decisions. Cleaning and preparing medical imaging data often includes outlier detection, removal of incomplete data, imputation, and deidentification (metadata and pixel-level PHI). The latter is the focus of the current study. Imaging data sets contain both relevant and irrelevant information (e.g., redundant, private). Figure 1: Typical example of an echocardiogram with a synthetic PHI overlay The aims of the current work are threefold: 1) Provide a framework for detecting and extracting PHI-based text from medical images and increasing security; 2) Apply and compare various methods of inpainting for redacted areas of images to meaningfully reconstruct lost pixel information and 3) Compare the influence the reconstruction techniques have on downstream supervised classification tasks. 1.1 State-of-the-Art Due to the risk of unauthorized sharing of protected health information (PHI), efforts have been made to de-identify medical image pixel and metadata for use in broader applications. For example, Newhauser et al. proposed to automate DICOM de-identification using Optical Character Recognition (OCR) to detect and mask all text within the image arrays [ 26 ] . Another previously approach to text masking uses a Convolutional Recurrent Neural Network (CRNN). This architecture is a combination of a Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Connectionist Temporal Classification (CTC) and is used for image-based sequence recognition tasks for text detection [ 38 ] . More generally, attributes within the image such as curves, lines, intersections, and loops, and how they occur in sequence relative to one another at different scales are often used for this purpose. For example, Fezai et al. applied a convolution neural network architecture using a simple auto-encoder to classify magnetic resonance imaging (MRI) equipment [ 6 ] from the metadata contained in the image header. Monteiro et al. proposed a convolution neural network (CNN) pipeline for de-identification of ultrasound DICOM images that removes identifying text only, while leaving non-identifying text intact [ 24 ] . Their software is available as a software-as-a-service, with an anonymization success rate of 89.2%. Huang et al. presented a method following HIPAA privacy rules, which also implemented an OCR [ 10 ] with a recorded success rate of 65%. Their algorithm, however, struggled to distinguish characters such as "Bg", from "B9" or "B1 from "Bl" and "B0 from "Bo". This may be important depending on downstream NLP-based filtering tasks. Oftentimes this is a difficult task due to nuisance features arising from the noisy medical image background, or text that ovarlies anatomical structures, which complicates the OCR methodology. However, completely de-identifying images is not always practical or needed, depending on the secondary application [ 28 ] . Therefore, the aim of de-identification should be to retain meaningful metadata ( e.g., age, race, sex in separate files in addition to crosswalks) and pixel (anatomical) information to allow for translation applications, while reducing PHI-induced security risks. Building on the methodologies and limitations of previous work, we also make use of OCR for text detection with respect to pixel information. However, unlike previous work, we hypothesize that removing and sequestering all text to a separate file structure provides a two-fold advantage. First, by removing all text, we increase assurance that PHI is, in fact, removed from the image content. Second, it places relevant patient information in a format that is easily accessible to end users. Second, extracting ’burned-in’ redundant information (as it is repeated in every frame) from the image array permits the truncation of pertinent image information. For example, oftentimes in ultrasound images text is repeated across frames. This process also decreases the amount of redundant information stored within the pixels, which is an expensive data format, exchanging it for data stored in tabular format. Further, downstream deep learning models may exploit any available text as a shortcut, creating spurious correlations. Pixel-level redaction of PHI is an essential requirement for protecting privacy in images, particularly in medical imaging. However, naïvely masking pixels (for example, by black boxes or heavy blurring) can affect local intensity distributions, textures, and anatomical boundaries in ways that degrade the performance of downstream computer vision models via shortcuts and biases. However, blurred images may be invertible using super-resolution-like methods. Generally, previous research on visual privacy preservation has shown that there is a fundamental privacy–utility trade-off: stronger obfuscation often comes at the cost of reduced recognition or segmentation accuracy unless the transformation is explicitly designed to preserve task-relevant image content [ 30 ] . In radiation oncology, for instance, systematic evaluation of defacing

Read the complete text