DuoFormer: Leveraging Hierarchical Representations by Local and Global Attention Vision Transformer

Xiaoya Tang 1,2, Bodong Zhang 1,3, Man Minh Ho 1, Beatrice S. Knudsen 4, Tolga Tasdizen 1,3 · 2025

Read DuoFormer: Leveraging Hierarchical Representations by Local and Global Attention Vision Transformer by Xiaoya Tang 1,2, Bodong Zhang 1,3, Man Minh Ho 1, Beatrice S. Knudsen 4, Tolga Tasdizen 1,3 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-12982.txt.

Description

Xiaoya Tang and four co-authors blend convolutional and transformer strengths for medical image diagnosis. Their paper appeared in two thousand twenty-five. Transformers capture broad context but need huge training sets. Convolutional networks carry useful biases and work at many scales. Medical images reward both local detail and global shape. Their DuoFormer stacks the two in a hierarchy.

A convolutional backbone first builds features at several scales. A fresh tokenization turns those features into transformer inputs. The multi-scale biases survive the handoff. A scale-wise attention mechanism then reads across levels directly. Hierarchical representation is the design's core claim. Computer-aided diagnosis is the target application. The model aims for accuracy without giant datasets.

Catalog record

CREATORXiaoya Tang 1,2, Bodong Zhang 1,3, Man Minh Ho 1, Beatrice S. Knudsen 4, Tolga Tasdizen 1,3
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-12982

Browse the full library · About the Tower of Babel