ABRA: Agent Benchmark for Radiology Applications

Bulat Maksudov, Vladislav Kurenkov, Kathleen M. Curran, Alessandra Mileo · 2026

Read ABRA: Agent Benchmark for Radiology Applications by Bulat Maksudov, Vladislav Kurenkov, Kathleen M. Curran, Alessandra Mileo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11224.txt.

Description

Bulat Maksudov and three colleagues introduce ABRA, an agent benchmark for radiology applications. Existing medical-agent benchmarks hand agents pre-selected images, never an environment to navigate. ABRA hands the agent control of an OHIF viewer plus an Orthanc DICOM server, worked through twenty-one function-calling tools.

Tools span slice navigation, windowing, series selection, pixel-coordinate annotation, and structured reporting. ABRA holds 655 generated tasks across three difficulty tiers and eight types, drawn from LIDC-IDRI, Duke Breast Cancer MRI, and NLST New-Lesion LongCT. Automatic scorers grade each episode on Planning, Execution, and Outcome. Ten models, five closed and five open-weight, reach at least 89 percent Execution on real annotation but only 0 to 25 percent Outcome. On the paired oracle variant with a simulated detector, Outcome reaches 69 to 100 percent. The gap points to perception, not tool orchestration, as the bottleneck. Builders get a benchmark testing navigation, not just answers.

Catalog record

CREATORBulat Maksudov, Vladislav Kurenkov, Kathleen M. Curran, Alessandra Mileo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91183
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.11224

Browse the full library · About the Tower of Babel