ABRA: Agent Benchmark for Radiology Applications
Bulat Maksudov, Vladislav Kurenkov, Kathleen M. Curran, Alessandra Mileo · 2026
Read ABRA: Agent Benchmark for Radiology Applications by Bulat Maksudov, Vladislav Kurenkov, Kathleen M. Curran, Alessandra Mileo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11224.txt.
Description
Bulat Maksudov and three colleagues introduce ABRA, an agent benchmark for radiology applications. Existing medical-agent benchmarks hand agents pre-selected images, never an environment to navigate. ABRA hands the agent control of an OHIF viewer plus an Orthanc DICOM server, worked through twenty-one function-calling tools.
Tools span slice navigation, windowing, series selection, pixel-coordinate annotation, and structured reporting. ABRA holds 655 generated tasks across three difficulty tiers and eight types, drawn from LIDC-IDRI, Duke Breast Cancer MRI, and NLST New-Lesion LongCT. Automatic scorers grade each episode on Planning, Execution, and Outcome. Ten models, five closed and five open-weight, reach at least 89 percent Execution on real annotation but only 0 to 25 percent Outcome. On the paired oracle variant with a simulated detector, Outcome reaches 69 to 100 percent. The gap points to perception, not tool orchestration, as the bottleneck. Builders get a benchmark testing navigation, not just answers.
Catalog record
| CREATOR | Bulat Maksudov, Vladislav Kurenkov, Kathleen M. Curran, Alessandra Mileo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 91183 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.11224