Ask the Expert: Collaborative Inference for Vision Transformers with Near-Edge Accelerators

Hao Liu, Suhaib A. Fahmy · 2026

Read Ask the Expert: Collaborative Inference for Vision Transformers with Near-Edge Accelerators by Hao Liu, Suhaib A. Fahmy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13334.txt.

Description

Vision transformers are too heavy for most edge devices, while sending everything to the cloud adds delay. This framework splits the job: a light generalist transformer runs on the edge device, and several medium expert transformers wait on a near-edge accelerator. A router reads the edge model's top predictions and, when confidence is low, hands the sample to the most relevant expert, whose training is progressively specialized on its subset.

On CIFAR-100 with a real edge and near-edge testbed, expert accuracy on target subsets rose 4.12 percent and overall accuracy rose 2.76 percent over static experts. Latency fell up to 45 percent against edge-only execution, and energy fell up to 46 percent against near-edge-only offload. Hao Liu leads the work with one co-author.

Catalog record

CREATORHao Liu, Suhaib A. Fahmy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-13334

Browse the full library · About the Tower of Babel