Ask the Expert: Collaborative Inference for Vision Transformers with Near-Edge Accelerators
Hao Liu, Suhaib A. Fahmy · 2026
Read Ask the Expert: Collaborative Inference for Vision Transformers with Near-Edge Accelerators by Hao Liu, Suhaib A. Fahmy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13334.txt.
Description
Vision transformers are too heavy for most edge devices, while sending everything to the cloud adds delay. This framework splits the job: a light generalist transformer runs on the edge device, and several medium expert transformers wait on a near-edge accelerator. A router reads the edge model's top predictions and, when confidence is low, hands the sample to the most relevant expert, whose training is progressively specialized on its subset.
On CIFAR-100 with a real edge and near-edge testbed, expert accuracy on target subsets rose 4.12 percent and overall accuracy rose 2.76 percent over static experts. Latency fell up to 45 percent against edge-only execution, and energy fell up to 46 percent against near-edge-only offload. Hao Liu leads the work with one co-author.
Catalog record
| CREATOR | Hao Liu, Suhaib A. Fahmy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-13334