SQUAD: Scalable Quorum Adaptive Decisions via ensemble of early exit neural networks
Matteo Gambella, Fabrizio Pittorino, Giuliano Casale, Manuel Roveri · 2026
Read SQUAD: Scalable Quorum Adaptive Decisions via ensemble of early exit neural networks by Matteo Gambella, Fabrizio Pittorino, Giuliano Casale, Manuel Roveri (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22711.txt.
Introduction
Matteo Gambella, Fabrizio Pittorino, Giuliano Casale, and Manuel Roveri built SQUAD, a team-vote system on early-exit networks. Early-exit nets give a quick answer mid-way when they feel sure. Their 2026 preprint notes that a lone model's sense of sure is often off, thanks to its own quirks.
SQUAD pools a quorum of early exits, adapting the bar as it goes. A team call is steadier than a lone hunch. The gain is speed where safe, care where not. The work serves live systems that must answer fast. It borrows wisdom from voting theory.
Catalog record
| CREATOR | Matteo Gambella, Fabrizio Pittorino, Giuliano Casale, Manuel Roveri |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22711
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SQUAD: Scalable Quorum Adaptive Decisions via ensemble of early exit neural networks Matteo Gambella, Fabrizio Pittorino, Giuliano Casale, Manuel Roveri Abstract Early-exit neural networks have become popular for reducing inference latency by allowing intermediate predictions when sufficient confidence is achieved. However, standard approaches typically rely on single-model confidence thresholds, which are frequently unreliable due to inherent calibration issues. To address this, we introduce SQUAD (Scalable Quorum Adaptive Decisions), the first inference scheme that integrates early-exit mechanisms with distributed ensemble learning, improving uncertainty estimation while reducing the inference time. Unlike traditional methods that depend on individual confidence scores, SQUAD employs a quorum-based stopping criterion on early-exit learners, by collecting intermediate predictions incrementally in order of computational complexity until a consensus is reached and halting the computation at that exit if the consensus is statistically significant. To maximize the efficacy of this voting mechanism, we also introduce QUEST (Quorum Search Technique), a Neural Architecture Search method to select early-exit learners with optimized hierarchical diversity, ensuring learners are complementary at every intermediate layer. This consensus-driven approach yields statistically robust early exits, improving the test accuracy up to 5.95% compared to state-of-the-art dynamic solutions with a comparable computational cost and reducing the inference latency up to 70.60% compared to static ensembles while maintaining a good accuracy. 1 Introduction Deep Neural Networks (DNNs) have established themselves as the de facto standard for many applications such as computer vision tasks, natural language processing, audio processing, and time series forecasting ( 3 ) . However, their remarkable performance typically comes at the cost of massive parameter counts and computational complexity ( 16 ) . This trade-off presents a critical bottleneck for deployment in resource-constrained environments, such as IoT sensors and Edge computing platforms, where energy budgets are strictly limited and low latency is paramount ( 1 ) . To mitigate these costs, the research community has increasingly adopted Dynamic Neural Networks for their ability to adapt their parameters and architecture according to the difficulty of the input. While these networks show many benefits, such as the increased efficiency, the mitigation of overfitting ( 34 ) , overthinking ( 18 ; 32 ) , and vanishing gradient phenomena ( 24 ) , many challenges are still open. In this paper, we specifically focus on a promising subset of this family, that is, Early Exit Neural Networks (EENNs) ( 30 ) . These architectures attempt to break the rigid one-size-fits-all computation paradigm by allowing easy-to-classify samples to exit the network at intermediate layers via auxiliary classifiers, denoted as Early Exit Classifiers (EECs), bypassing the deeper blocks ( 28 ; 23 ) . However, standard EENNs suffer from a fundamental reliability issue: the decision to exit is typically governed by a heuristic confidence threshold (e.g., softmax entropy) derived from a single model. Since DNNs are notoriously prone to overconfidence, even when making erroneous predictions, relying on a single agent self-assessment often leads to risky premature exits or unnecessary computation ( 31 ; 14 ) . To address this shortcoming, we propose a shift to a collective confidence in early exits. Drawing inspiration from Ensemble Learning to reduce variance ( 21 ) , we introduce SQUAD (Scalable Quorum Adaptive Decisions) that combines early exits with the ensemble technique. Instead of relying on the confidence of a single classifier, we query a dynamic committee of early-exit learners to decide whether to stop computation both horizontally (along the learners) and vertically (along the early exits). Indeed, in our framework, computation is halted horizontally, invoking each learner incrementally in order of computational complexity upon reaching a majority consensus, and vertically if the early exit decision is triggered once statistical robustness of the consensus is ensured through a t-test. This approach enhances the early exit decision with a statistically robust voting process, effectively mitigating the calibration issues of single-model EENNs. While conceptually appealing, implementing SQUAD is non-trivial. For a voting mechanism to be efficient, the committee members must offer complementary predictions not just at the final output, but at every intermediate early exit gate. We can define this property as hierarchical diversity . This aspect is crucial since if all learners share the same biases or errors at layer L , the quorum will merely reinforce a wrong decision, and this is more likely to happen at the very early layers. Designing such a system manually is infeasible. Consequently, we could resort to Neural Architecture Search ( 25 ) , a method that is attracting steadily growing research interest for its ability to automatically design the architectures of NNs given a task and a dataset. Unlike traditional NAS, which optimizes for individual accuracy ( 9 ; 10 ; 33 ) , or ensemble methods that seek diversity only at the output ( 35 ; 29 ) , our approach explicitly searches for a set of learners that maximizes hierarchical diversity . We term this unified framework QUEST (Quorum Ensemble Search Technique). By effectively combining the efficiency of early exits with the robustness of ensembles through a novel NAS strategy, QUEST offers a unique solution to the accuracy-efficiency dilemma. Lastly, empirical results on CIFAR-10 ( 19 ) , CIFAR-100 ( 19 ) , and ImageNet16-120 ( 6 ) demonstrate that QUEST achieves a superior balance between accuracy and computational cost. Specifically, we improve inference time up to 70.60% compared to static ensembles and increase accuracy up to 5.95% over standard confidence-based EENNs at comparable latency. To facilitate comparisons and reproducibility, the source code of QUEST is released to the scientific community as a public repository 1 1 1 URL to be disclosed after paper acceptance . 2 Background 2.1 Early Exit Neural Networks Early Exit Neural Networks (EENNs), which allow intermediate predictions when enough confidence is achieved, are used to reduce the computational cost of deep neural networks ( 28 ) . An EENN can be viewed as a composition of b sequential blocks, f(x)=\mathrm{B}_{b}\circ\cdots\circ\mathrm{B}_{1}(x) , where a classification head C_{i} is attached to each block: f_{i}(x)=\mathrm{C}_{i}\circ\left(\mathrm{B}_{i}\circ\cdots\circ\mathrm{B}_{1}(x)\right) (1) This formulation produces a sequence of predictions, allowing the system to halt computation once a consensus or confidence threshold is met. Training is typically performed by jointly updating all exits: L_{\text{joint}}(x,y)=l(f_{b}(x),y)+\sum_{i=1}^{b-1}\beta_{i}\cdot l(f_{i}(x),y) (2) where \beta_{i} weights the i -th branch loss and x is the data input. A weakness of this technique is that confidence thresholds are often unreliable due to calibration issues ( 31 ; 14 ) . A metric of calibration is the Expected Calibration Error (ECE) ( 22 ) , which measures the reliability of a model by quantifying the expected difference between its predicted confidence and its actual accuracy. To calculate this, the full probability range is divided into M disjoint bins (i.e., intervals of size 1/M ), and samples are grouped into these bins based on their predicted confidence scores. For each bin B_{m} , the average accuracy \text{acc}(B_{m}) and average confidence \text{conf}(B_{m}) are computed. The final ECE score is the weighted average of the absolute difference between these two values, weighted by the fraction of samples \frac{|B_{m}|}{N} in each bin, as shown in the formula: \text{ECE}=\sum_{m=1}^{M}\frac{|B_{m}|}{N}\left|\text{acc}(B_{m})-\text{conf}(B_{m})\right| (3) 2.2 Ensemble Learning and Consensus A way to improve the calibration of deep neural networks is by Ensemble Learning ( 21 ) , where multiple