Dataset and Benchmark for Enhancing Critical Retained Foreign Object Detection

Yuli Wang, Victoria R. Shi, Liwei Zhou, Richard Chin, Yuwei Dai, Yuanyun Hu, Cheng-Yi Li, Haoyue Guan, Jiashu Cheng, Yu Sun, Cheng Ting Lin, Ihab Kamel, Premal Trivedi, Pamela Johnson, John Eng, Harrison Bai · 2025

Read Dataset and Benchmark for Enhancing Critical Retained Foreign Object Detection by Yuli Wang, Victoria R. Shi, Liwei Zhou, Richard Chin, Yuwei Dai, Yuanyun Hu, Cheng-Yi Li, Haoyue Guan, Jiashu Cheng, Yu Sun, Cheng Ting Lin, Ihab Kamel, Premal Trivedi, Pamela Johnson, John Eng, Harrison Bai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06937.txt.

Introduction

After surgery, a sponge or needle left inside a patient is a rare but grave error, and finding such items on chest X-rays is a task where computer help could save lives. Yuli Wang and co-authors built the first large test set for that job in a 2025 paper. It holds chest films with true leftover items gathered over many years at one hospital system.

The team ran leading object-finding models on the set and found clear room to grow. Since real cases are scarce, they also tested two ways to fake more films: one based on physics, one on picture-making models. Each fake style had strengths and gaps, which the paper maps in detail. The set and scores give the field a shared starting point for safer scans.

Catalog record

CREATORYuli Wang, Victoria R. Shi, Liwei Zhou, Richard Chin, Yuwei Dai, Yuanyun Hu, Cheng-Yi Li, Haoyue Guan, Jiashu Cheng, Yu Sun, Cheng Ting Lin, Ihab Kamel, Premal Trivedi, Pamela Johnson, John Eng, Harrison Bai
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-06937

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dataset and Benchmark for Enhancing Critical Retained Foreign Object Detection Yuli Wang, Victoria R. Shi, Liwei Zhou, Richard Chin, Yuwei Dai, Yuanyun Hu, Cheng-Yi Li, Haoyue Guan, Jiashu Cheng, Yu Sun, Cheng Ting Lin, Ihab Kamel, Premal Trivedi, Pamela Johnson, John Eng, Harrison Bai Abstract Critical retained foreign objects (RFOs), including surgical instruments like sponges and needles, pose serious patient safety risks and carry significant financial and legal implications for healthcare institutions. Detecting critical RFOs using artificial intelligence remains challenging due to their rarity and the limited availability of chest X-ray datasets that specifically feature critical RFOs cases. Existing datasets only contain non-critical RFOs, like necklace or zipper, further limiting their utility for developing clinically impactful detection algorithms. To address these limitations, we introduce "Hopkins RFOs Bench", the first and largest dataset of its kind, containing 144 chest X-ray images of critical RFO cases collected over 18 years from the Johns Hopkins Health System. Using this dataset, we benchmark several state-of-the-art object detection models, highlighting the need for enhanced detection methodologies for critical RFO cases. Recognizing data scarcity challenges, we further explore image synthetic methods to bridge this gap. We evaluate two advanced synthetic image methods—DeepDRR-RFO, a physics-based method, and RoentGen-RFO, a diffusion-based method—for creating realistic radiographs featuring critical RFOs. Our comprehensive analysis identifies the strengths and limitations of each synthetic method, providing insights into effectively utilizing synthetic data to enhance model training. The Hopkins RFOs Bench and our findings significantly advance the development of reliable, generalizable AI-driven solutions for detecting critical RFOs in clinical chest X-rays. Dataset and benchmark code are open at Github https://github.com/YuliWanghust/RFO_Bench 1 Introduction Retained foreign objects (RFOs), such as sponges, needles, sutures, and other instruments, can have severe consequences for patients, leading to significant financial and legal repercussions for the involved medical institutions. These reportable "never events" are rare, with studies estimating an incidence of approximately 1 to 2 in 10,000 surgeries, with rates as high as 1 in 1000 to 1500 for abdominal surgeries lincourt2007retained ; zejnullahu2017retained . Although RFOs are typically discussed within the context of a surgical procedure, they can also occur after non-surgical interventions. For example, the incidence of retained guidewires following central line placement is estimated at 0.03% mariyaselvam2022central ; vannucci2013retained . The actual number of RFO cases is likely underreported due to low reporting rates, and many patients may be asymptomatic, remaining unaware of the event. Recently, AI has emerged as a promising approach to improving RFOs detection in radiographs, with potential benefits in clinical workflow optimization kawakubo2023deep ; asiyanbola2012modified ; kufel2023chest ; santosh2021generic ; wang2024car . Despite encouraging results from deep neural networks, several critical limitations remain toruner2024artificial ; wang2024enhancing . Foremost is the scarcity of publicly available datasets; most intraoperative radiographs obtained for procedural guidance are not routinely archived, limiting the development of large-scale datasets. Moreover, existing public datasets (shown in Table 1 ) only include non-critical RFOs, such as external instruments, tubes, and zippers, which pose no patient risk. Conversely, critical RFOs like needles and surgical sponges, associated with severe complications, are rarely captured due to their infrequent occurrence, limiting the medical relevance of current datasets. These challenges highlight the urgent need for a large dataset with critical RFOs, new solutions to address data constraints, reliably detect both critical and non-critical RFOs, and ensure generalizability across diverse clinical settings. Moreover, to overcome the limitation posed by insufficient data, various synthetic radiograph generation methods have been proposed, including physics-based methods such as virtual radiograph generators, and denoising diffusion probabilistic model (DDPM). Physics-based methods simulate realistic radiographs by employing physical modeling of anatomical structures and X-ray interactions, allowing for controlled experimentation with various object placements and imaging conditions. On the other hand, DDPM-based methods generate synthetic images by progressively denoising randomly sampled images, enabling the creation of diverse and realistic data without explicit physical modeling. However, both approaches are significantly constrained by the extreme rarity of critical RFOs cases, and thus far, none have incorporated critical RFOs into their synthetic scope, limiting their practical application for training robust AI algorithms for critical RFOs detection. Addressing these significant challenges in AI-driven RFOs detection on radiographs, we present the following contributions in this paper: • Development of "Hopkins RFOs Bench," the first and largest dataset of its kind , comprising 144 chest X-ray images containing critical RFOs collected from the Johns Hopkins Health System over the past 18 years. • Benchmarking existing object detection models on our proposed dataset to underscore the necessity for advanced detection methods. • Evaluating two customized synthetic image generation models, DeepDRR-RFO and RoentGen-RFO, for creating images with critical RFOs. We train object detection models using these synthetic images, analyze the strengths and weaknesses of each approach, and provide insights to guide future improvements utilizing our openly accessible dataset. 2 Related Works 2.1 Clinical Safety Mechanism for RFOs incidence Given the medical complications and medicolegal implications of RFOs, several safety mechanisms have been established to reduce their incidence webb1988management . In surgical procedures, operating room staff routinely perform final counts to ensure that no foreign objects are left inside the patient. If a discrepancy in the final counts arises, intraoperative radiographs are obtained to detect any RFOs. However, even with these safety measures, more than 80% of surgeries involving RFOs have reported correct counts at the conclusion of the procedure gibbs2007preventable . When clinical teams fail to detect RFOs during the procedure, they are often discovered incidentally during follow-up imaging studies takahashi2023characteristics . Radiologists, however, may miss these objects because they do not usually expect to find them during routine imaging interpretation. Regarding imaging techniques for RFOs detection, plain radiographs remain the gold standard. On X-ray, RFOs often appear as radiopacities with associated mass effects, mottled air, or density changes in surrounding soft tissues kumar2017imaging . Most RFOs contain radiopaque markers, which enhance their visibility on X-ray. However, these markers can become disfigured within the body, reducing their reliability o2003imaging . RFOs without radiopaque markers are often identified through cross-sectional imaging or by detecting radiolucency caused by air trapping yun2021retained . It is important to note that false-negative radiographs do occur, with some studies suggesting that intraoperative radiographs may miss up to one-third of RFOs cima2008incidence . Additionally, radiologists interpreting these radiographs often face time pressures, particularly after surgery, which can increase the likelihood of errors. 2.2 Deep Learning Based RFOs Detection and Its Dataset Kawakubo et al. kawakubo2023deep developed a deep learning model that effectively detected retained surgical items through the post-processing of fused images containing surgical sponges and unremarkable postoperative X-rays. In a proof-of-concept study, Asiyanbola et al. asiyanbola2012modified employed a map-seeking circuit for the detection of needles within abdominal X-rays. Similarly, Kufel et al. kufel2023chest designed a convolutional neural network (CNN) to identify objects in chest X-ray images using the publicly available National Institutes of Health Chest X-ray (CXR) dataset wang2017chestx . Additionally, Santosh et al. santosh2021generic ; zhou2020automatically applied the YOLOv4 model for non-critical RFOs detection on the same public CXR dataset. Although CNNs have demonstrated promising capabilities for detecting and identifying RFOs, several limitations persist. A significant barrier to AI-driven RFO detection is the scarcity of publicly available datasets. Most intraoperative radiographs taken for procedural guidance are not archived, and existing

Read the complete text