DuoFormer: Leveraging Hierarchical Representations by Local and Global Attention Vision Transformer
Xiaoya Tang 1,2, Bodong Zhang 1,3, Man Minh Ho 1, Beatrice S. Knudsen 4, Tolga Tasdizen 1,3 · 2025
Read DuoFormer: Leveraging Hierarchical Representations by Local and Global Attention Vision Transformer by Xiaoya Tang 1,2, Bodong Zhang 1,3, Man Minh Ho 1, Beatrice S. Knudsen 4, Tolga Tasdizen 1,3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-12982.txt.
Description
Xiaoya Tang and four co-authors blend convolutional and transformer strengths for medical image diagnosis. Their paper appeared in two thousand twenty-five. Transformers capture broad context but need huge training sets. Convolutional networks carry useful biases and work at many scales. Medical images reward both local detail and global shape. Their DuoFormer stacks the two in a hierarchy.
A convolutional backbone first builds features at several scales. A fresh tokenization turns those features into transformer inputs. The multi-scale biases survive the handoff. A scale-wise attention mechanism then reads across levels directly. Hierarchical representation is the design's core claim. Computer-aided diagnosis is the target application. The model aims for accuracy without giant datasets.
Catalog record
| CREATOR | Xiaoya Tang 1,2, Bodong Zhang 1,3, Man Minh Ho 1, Beatrice S. Knudsen 4, Tolga Tasdizen 1,3 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-12982