Adaptive Multi-Expert Reasoning via Difficulty-Aware Routing and Uncertainty-Guided Aggregation

Mohamed Ehab, Ali Hamdi · 2026

Read Adaptive Multi-Expert Reasoning via Difficulty-Aware Routing and Uncertainty-Guided Aggregation by Mohamed Ehab, Ali Hamdi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10335.txt.

Introduction

Ehab and Hamdi describe Adaptive Multi-Expert Reasoning, a scheme that adapts language-model thought to problem hardness. A routing system reads the problem text, predicts hardness and doubt, and guides sampling breadth. Three experts draft tries, refined through fix and final phases. A neural checker checks truth, while grouping-based picking chooses the final answer from accord and quality. On GSM8K, the system scores about 75 percent using only first training data, beating most similar 7B models trained on fake data.

Catalog record

CREATORMohamed Ehab, Ali Hamdi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20784
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10335

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adaptive Multi-Expert Reasoning via Difficulty-Aware Routing and Uncertainty-Guided Aggregation Mohamed Ehab, Ali Hamdi Abstract Large language models (LLMs) demonstrate strong performance in math reasoning benchmarks, but their performance varies inconsistently across problems with varying levels of difficulty. This paper describes Adaptive Multi-Expert Reasoning (AMR), a framework that focuses on problem complexity by reasoning with dynamically adapted strategies. An agile routing system that focuses on problem text predicts problems’ difficulty and uncertainty and guides a reconfigurable sampling mechanism to manage the breadth of generation. Three specialized experts create candidate responses, which are modified during multiple correction and finalization phases. A neural verifier assesses the correctness of responses, while a clustering-based aggregation technique identifies the final candidate answer based on a combination of consensus and answer quality. When evaluated on the GSM8K dataset, AMR achieved 75.28% accuracy while only using the original training data. This result outperformed the majority of comparable 7B models that were trained on synthetic data. This showcases that models using difficulty-based routing and uncertainty-driven aggregation are efficient and effective in improving math reasoning models’ robustness. I Introduction There has been a lot of recent success with large language models (LLMs) and their ability to reason with mathematics, particularly with newly developed benchmarks like GSM8K [ 1 ] . A lot of positive work has also been done with scaling, instruction tuning, and reasoning via chain-of-thought [ 15 ] for better performance with multi-step reasoning tasks. LLMs, however, still show a lot of variability with the problems that have different levels of reasoning complexity. They end up struggling to generalize, whether it be a simple arithmetic calculation, or a more difficult multi-step reasoning problem. Existing approaches like uniform prompting, a way to solve a problem without reasoning, and static ensemble averaging outputs, without problem difficulty and expert specialization, lack flexibility with 2 observations that must be considered: (1) more than one reasoning style is needed, and (2) problems can have varying levels of underlying complexity which can be difficult to solve.In this paper, we present AMR, a reasoning framework with multi-experts that is also aware of the problem’s difficulty, to solve these reasoning problems. The framework has 4 main components: (i) a difficulty-aware router, (ii) multiple stylistically specialized experts, (iii) a neural verifier, and (iv) a clustering-based aggregation mechanism. Fig. 1: AMR processes each problem based on predicted difficulty and uncertainty, routes them to various potential solution generators in different expert categories, and guides answer selection via correction and finalization to answer clusters as determined by a verifier. Our method has notable similarities to mixture-of-experts (MoE) architectures [ 12 , 2 ] , though they are fundamentally different because they perform routing at inference time across reasoning styles, as opposed to through learned parameter sparsity in a single model. The router predicts both difficulty and uncertainty, which gives it the ability to control generation diversity adaptively. While in training and evaluation, the problem difficulty is estimated via a heuristic based on the number of reasoning steps in the reference solution (this serves as a stand-in for gold difficulty), during inference, the router model solely predicts difficulty from the problem text. This is to ensure that routing choices are made based on test-time evidence only. Three reasoning-expert models based on LoRA generate different reasoning paradigms. They provide additional candidates through correction and finalization passes, thus enhancing robustness. One verifier evaluates the candidates for correctness and a clustering-based aggregation method evaluates and selects the final output based on agreement and quality. Unlike most existing approaches, AMR does not depend on extensive synthetic data or enormous model size increases. Instead, it enhances reasoning through adaptive inference decision making. We identify four key contributions. First, we implement a difficulty-sensitive routing mechanism featuring uncertainty estimation that directly manages the number and variety of reasoning approaches generated.We also present a multi-expert reasoning framework that incorporates stylistically specialized LoRA-adapted experts, as well as a correction step that improves erroneous answers and a finalization step that delivers clear and high-quality outputs. Third, we build a clustering-based aggregation approach that combines verifier confidence, answer quality, and expert consensus on how to reliably choose a final answer. To conclude, we show that our method outperforms competitors on data efficiency by achieving strong performance (75.28% accuracy on GSM8K) using only the original training data. This is in contrast to other methods that depend on extensive, synthetic data augmentation. II Related Work II-A Robustness Benchmarks and the GSM-PLUS Study GSM8K [ 1 ] is a widely accepted metric to measure mathematical reasoning abilities of models. It has, however, been shown that models that achieve high accuracy on these tasks, still fail when the data is altered. Li et al. [ 7 ] present GSM-PLUS, a more complete benchmark with problem alterations from multiple directions (linguistic variations, numerical variations, alterations in the solution space). Here we present their most notable findings: • Closed-source models: GPT-4 sees an 8.23% absolute drop from 93.25% on GSM8K to 85.58% on GSM-PLUS; GPT-3.5 has a 16.88% drop from 73.62% to 61.19%. • Open-source base models: Mistral-7B sees a drop from 39.58% to 26.18%; LLaMA-2-7B from 13.42% to 8.12%; LLaMA-2-70B from 56.71% to 40.04%. • Math-specialized models: Even with fine-tuning on synthetic data, MetaMath-7B drops from 66.72% to 44.35%; ToRA (7B) from 67.48% to 43.60%; MAmmoTH (7B) from 52.84% to 32.14%. Robustness yet is a crucial concern, and this is what triggered our emphasis on uncertainty modeling, and difficulty-aware routing. II-B Data Scaling and Verifier-Based Selection An alternative line of research examines the limits of small models trained with plentiful synthetic data. Approaches like ReAct [ 16 ] combine reasoning and external tool usage. Augmented problem-solving abilities have been shown to be the result of operational language models. Program-aided methods like PAL [ 3 ] use symbolic execution to enhance their numerical precision, and this is seen as a neural reasoning alternative. Liu et al. [ 8 ] introduce TinyGSM, a dataset of 12.3 million grade school math problems with corresponding Python solutions derived from a superior teacher model. After refinement on TinyGSM, a pair of Phi-1.5 models (1.3B each) – a generator and a verifier – achieves 81.5% accuracy on GSM8K, surpassing significantly larger baselines. Some of the most important results of their work includes: • Baselines: GPT-3.5 (77.4%), GPT-4 (97.0%); LLaMA-2-7B (14.6%), LLaMA-2-70B (56.8%), Mistral-7B (52.2%). • Math-tuned models: MetaMath-7B (66.5%), MetaMath-70B (82.3%), WizardMath-70B (81.6%), ToRA-Code-70B (84.3%). • TinyGSM progression: Phi-1.5 (44.6%), Phi-GSM (68.2%), Phi-GSM+V (81.5%). These findings demonstrate the advantages of integrating large-scale synthetic data with verifier-guided selection, even though the data needs are high. According to [ 14 ] , self-consistency decoding improves results through the use of different reasoning pathways and the selection of consistent results. In contrast, AMR is data efficient, achieving 75.28% accuracy with only the original GSM8K dataset. In addition, AMR is built on a 7B-scale model, whereas closed-source models like GPT-3.5 and GPT-4 are assumed to operate on a significantly higher level, often in the tens to hundreds of billions of parameters. This model capacity variation should be taken into account in performance comparisons. II-C Enhanced Problem Understanding (DUP) Zhong et al. [ 19 ] state that chain-of-thought prompting has issues with semantics. As a result, they created

Read the complete text