Primus: Enforcing Attention Usage for 3D Medical Image Segmentation
Tassilo Wald tassilo.wald (at) dkfz-heidelberg.de, Saikat Roy contact (at) saikatroy.me, Fabian Isensee f.isensee (at) dkfz-heidelberg.de, Constantin Ulrich, Sebastian Ziegler, Dasha Trofimova, Raphael Stock, Michael Baumgartner Gregor Köhler, Klaus Maier-Hein · 2025
Read Primus: Enforcing Attention Usage for 3D Medical Image Segmentation by Tassilo Wald tassilo.wald (at) dkfz-heidelberg.de, Saikat Roy contact (at) saikatroy.me, Fabian Isensee f.isensee (at) dkfz-heidelberg.de, Constantin Ulrich, Sebastian Ziegler, Dasha Trofimova, Raphael Stock, Michael Baumgartner Gregor Köhler, Klaus Maier-Hein (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01835.txt.
Description
Transformer networks conquered language and images, yet plain convolutional networks still win at outlining organs in 3D medical scans. Tassilo Wald and co-authors asked why. They found that many so-called transformer models lean on their convolutional parts. Removing the transformer sometimes changed nothing.
Their answer is Primus, a design that forces attention to do the work, using fine-grained tokens and updated positional schemes. A second version refines how image patches enter the network. Across nine public datasets, the first Primus matched the standard champion network. The second version surpassed it and stood level with the best convolutional designs. The authors call Primus the first truly competitive transformer-first segmenter.
Catalog record
| CREATOR | Tassilo Wald tassilo.wald (at) dkfz-heidelberg.de, Saikat Roy contact (at) saikatroy.me, Fabian Isensee f.isensee (at) dkfz-heidelberg.de, Constantin Ulrich, Sebastian Ziegler, Dasha Trofimova, Raphael Stock, Michael Baumgartner Gregor Köhler, Klaus Maier-Hein |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 116 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01835