Adaptive confidence ensemble reranking for reliable knowledge-intensive question answering Authors: Tanzila Kehkashan, Maha Abdelhaq, Muhammad Abdullah, Sharifah Sakinah Syed Ahmad, Nor Azman Ismail, Nikola Ivković, Adnan Akunzada Published: 2026 ABSTRACT AbstractModern question answering systems rely on advanced neural architectures to bridge the semantic gap between natural language queries and relevant textual evidence. However, single-model approaches often struggle to capture diverse linguistic variations, while traditional ensemble methods suffer from high computational complexity and static model combination strategies. Existing answer reranking systems lack adaptive mechanisms to dynamically adjust model contributions based on question characteristics, limiting their effectiveness across knowledge-intensive tasks. This study proposes an adaptive confidence ensemble reranking framework designed to improve reliability and efficiency in knowledge-intensive question answering. The proposed approach integrates dual-encoder retrieval with a cross-encoder ensemble of BERT, RoBERTa, and DeBERTa models, combined through an instance-aware adaptive confidence weighting mechanism. The framework dynamically adjusts model contributions using cross-entropy-based evaluation to optimize answer ranking performance while maintaining computational feasibility. Experimental results demonstrate significant performance improvements across multiple benchmark datasets, including a 9.3% increase in mean average precision on MS MARCO, a 2.1% gain on WikiQA, and a 1.2% improvement on TREC-QA. These findings highlight the effectiveness of the proposed method in enhancing reliability and accuracy across diverse knowledge-intensive question answering scenarios.Keywords: Answer reranking, Cross-encoder ensemble, Adaptive confidence weighting, Question answering systems, Transformer modelsSubject terms: Engineering, Mathematics and computing Introduction Question answering systems have gone beyond the simple mechanism of searching using a keyword to complex neural networks that are able to comprehend intricate semantic links between queries and textual information. Contemporary QA systems tend to use a pipeline with multi-steps that include passage retrieval phase, answer extraction phase, and more recently, answer reranking phase, which improve on initial predictions to achieve a better system accuracy1. Modern systems exploit transformer based frameworks and dense retrieval schemes to overcome the semantic gap between queries in natural language and appropriate text emergent material2. reranking of answers is a crucial step of post-processing, which overcomes the drawback of solely single-pass retrieval and extraction systems by employing advanced scoring schemes to the candidate answers3. Recent developments in cross-encoder models have shown substantial enhancement over the traditional bi-encoder models by jointly computing question-answer pair similarities using transformer networks, which can compute similarities in a more refined manner4. Combining dense passage retrieval with large-scale semantic matching of millions of passages has transformed large-scale question answering by applying dense passage retrieval methods to large in-memory libraries of efficient vector similarity search techniques such as FAISS5. Knowledge distillation reduces large ensemble models into efficient single networks while preserving their performance benefits6. Although significant advancements have been made in answering reranking methods, the current techniques have a number of severe constraints limiting their application use. Conventional learning-to-rank frameworks are based on heavy reliance on handcrafted features and cannot represent complicated semantic connections between queries and responses7. Recent cross-encoder based reranking systems, despite achieving higher accuracy, are limited by quadratic computational complexity, which constrains their scalability in production settings8. Multi-teacher knowledge distillation methods are promising but fail to assign effective weights to the different teacher models, which can result in suboptimal transfer of knowledge9. In addition, existing ensemble approaches lack mechanisms to adapt model contributions based on question type or complexity10. Computational efficiency versus accuracy trade off is not fully determined, especially in the case of real time application where response time is required in the sub-second range11. The key problem with the modern question answering systems is that it is difficult to create an answer reranking framework at once being highly accurate and being computationally efficient enough to be deployed at large scales. Current single-model methods cannot capture diverse linguistic patterns in real-world questions, while existing ensemble techniques lack instance-sensitive aggregation schemes, leading to suboptimal answers for complex queries12–14. Our approach differs from related paradigms along three specific technical dimensions: (i) Unlike standard mixture-of-experts (MoE) frameworks, which apply static or input-independent gating functions, our aggregator conditions ensemble weights jointly on cross-encoder score vectors and a 4-dimensional question-type one-hot encoding making weighting instance-aware at the query level rather than at the input-feature level. (ii) Unlike multi-teacher knowledge distillation12, which compresses ensemble knowledge into a single student model and discards model diversity at inference time, our architecture retains all three cross-encoders during inference and dynamically adjusts their per-query contributions without any knowledge compression. (iii) Unlike RankT515, which is a single sequence-to-sequence reranker with a fixed encoder architecture, our framework explicitly combines three architectures (BERT16, RoBERTa17, DeBERTa18) with distinct pre-training corpora and attention factorizations DeBERTa’s disentangled attention and BERT’s bidirectional encoding capture complementary linguistic patterns that no single architecture can replicate. Formally, the problem addressed in this work is: given a question Q and a set of K candidate answers retrieved by a dual-encoder, how to optimally combine relevance scores from multiple cross-encoder models through an instance-aware weighting mechanism that adapts to question characteristics, overcoming the limitation of static ensemble strategies that apply fixed weights regardless of question type or complexity. The primary objectives of this research are structured as follows: i.To develop an ensemble-based answer reranking architecture that integrates multiple transformer models with an instance-aware adaptive confidence weighting mechanism which conditions ensemble weights on question-level characteristics, unlike existing methods that use static or fixed-weight combinations. To develop an ensemble-based answer reranking architecture that integrates multiple transformer models with an instance-aware adaptive confidence weighting mechanism which conditions ensemble weights on question-level characteristics, unlike existing methods that use static or fixed-weight combinations. ii.To design and implement an instance-aware weighting system that dynamically adjusts model contributions based on question characteristics and model expertise for optimal answer selection. To design and implement an instance-aware weighting system that dynamically adjusts model contributions based on question characteristics and model expertise for optimal answer selection. iii.To empirically validate the computational efficiency and scalability of the proposed framework as secondary properties across three benchmark datasets. To empirically validate the computational efficiency and scalability of the proposed framework as secondary properties across three benchmark datasets. This study introduces a new ensemble-based answer reranking model to address the computational limitations of current methods while improving accuracy through adaptive model combination. Unlike existing fusion strategies such as simple score averaging, fixed linear combination, or model stacking, our approach introduces an instance-aware confidence-weighting mechanism that conditions ensemble weights on question-level characteristics. Compared to multi-teacher knowledge distillation methods12, which compress ensemble knowledge into a single student model and lose model diversity at inference time, our architecture retains all cross-encoder models and dynamically adjusts their contributions per query. The confidence-weighting mechanism measures cross-entropy between the outputs of teacher models and ground truth distributions, reducing erroneous knowledge transfer and enhancing ensemble performance. The suggested architecture has shown significant advancements in ranking accuracy across benchmark datasets including MS MARCO, WikiQA, and TREC-QA, while maintaining computational efficiency suitable for production deployment. Moreover, our extensive ablation experiments with results reported over three random seeds provide empirical evidence for the effectiveness of each architectural component, offering clear design guidelines for ensemble-based question answering systems. The structure of the paper is as follows; Section "Literature review" gives a detailed literature review of available literature on answer reranking methods. Section "Methodology" gives the proposed methodology, which includes selecting the models, data collection and preprocessing, feature engineering, mathematical formulation, and architecture design. In Section "Experiments and results", the experimental findings are presented, ablation experiments and quantitative comparisons with state-of-the-art algorithms are provided and computational performance is analyzed. Section "Discussion" speaks on the importance of these findings, implications and limitations of the research. Section "Conclusion and future recommendations" conclude the study and provides recommendations on future research. This paper demonstrates that the answer reranking by the ensemble with adaptive confidence weighting can be presented as positively influence the performance of the question answering and be computationally efficient by the strategic combination of different transformer models and instance-sensitive aggregation. Literature review This literature review explores the development of answer reranking algorithms, starting with traditional machine learning methods, moving on to innovations of deep learning, and finally looking at transformer-based architectures. Machine learning foundations in answer reranking The paradigms of the foundational methods of answer reranking were based on the old paradigms of information retrieval, in which early systems used rule-based and feature-engineering methods to enhance answer selection19. Initial applications were based on lexical matching methods, and TF-IDF and BM25 scoring systems were used to generate candidate answers. Although such sparse approaches to retrieval were computationally efficient, they proved limited to a considerable degree in the ability to represent a semantic relationship between questions and candidate answers20. The idea of supervised learning as an answer candidate ordering method has been introduced through a range of traditional learning-to-rank systems, which form the conceptual basis of modern reranking systems21. Usually, these methods used support vector machines (SVM) and random forest algorithms and relied on handcrafted features, like lexical overlap, lexical matching of known entities, and syntactic similarity values22. The shift of the sparse to the dense representations was a crucial development in retrieval efficiency. Dense passage retrieval (DPR) proposed two-encoder systems that showed spectacular increases on the traditional sparse systems and reached an 9-19% overall increase in top-20 passage retrieval accuracy23. This theoretical breakthrough laid the groundwork of more state-of-the-art reranking methods with the ability to compute semantic similarity in continuous vectors spaces with efficient indexing algorithms such as FAISS2. Deep learning integration and neural architectures Answer reranking was reinvented by the introduction of more advanced representation learning abilities through the integration of deep neural networks24. Recurrent neural networks (RNN) and long short-term memory (LSTM) networks were the first networks able to make significant gains in terms of modeling orderly dependencies among pairs of questions and answers. These architectures dealt with the sequential property of textual data and was efficient computationally but was restricted by the inherent sequential processing that such architectures provided1. Convolutional neural networks (CNN) provided some complementary benefits as local n-gram pattern has been presented and the hierarchical feature representation25. The hybrid CNN-RNN architecture in hybrid models also exhibited superior functionality in a variety of QA benchmarks, especially in cases where both local pattern recognition and sequential modeling properties were needed26. Transformer-based models and cross-encoder architectures Transformer architectures were introduced and impacted answer reranking capabilities with self-attention and bidirectional context modeling in it fundamentally27. BERT-based cross-encoders showed better results than the traditional two-encoder models because they operated equally on pairs of question-answer and allowed a more subtle similarity calculation. Cross-encoders are more successful than bi-encoders because they can encode both input sequences at the cost of higher computational cost28. Recent trends in the optimization of cross-encoders are directed to the efficiency gains without compromising those. Methods like knowledge distillation can be used to reduce the size of large cross-encoders into smaller and more efficient models, with results of 2.4x model size and 7x runtime reduction and similar accuracy29. These optimizations overcome the main drawback of cross-encoders, which is their computational intensity within production settings8. Multi-encoder ensemble methods have also been integrated which has increased the effectiveness of reranking. Multiple transformer model systems (BERT, RoBERTa, DeBERTa) with adaptive confidence combination are uniform in their gain according to benchmark datasets30. These ensemble techniques exploit diversity of models in order to enhance robustness and overcome the weaknesses of single-model techniques to a variety of question types31. Recent work has further demonstrated the effectiveness of disentangled attention mechanisms for improving transformer-based language understanding32. Contemporary developments and emerging paradigms New studies have discussed the integration of large language models (LLMs) in reranking systems, and methods such as RankGPT have shown competitive performance using prompt-based reranking strategies11. Nevertheless, recent empirical evaluations demonstrate that fine-tuned cross-encoders and sequence-to-sequence rerankers remain competitive with LLM-based approaches while achieving substantially higher computational efficiency15. The idea of retrieval-augmented generation (RAG) integration is an important modern trend, with the reranking elements playing a crucial role as the links between retrieval and generation stages15. The current RAG systems use advanced reranking algorithms in order to guarantee the quality of retrieved context, which has a direct effect on the generation accuracy and the factual consistency33. Research gaps In spite of the significant progress, a number of significant gaps remain in the present-day answer reranking methods. The trade-off of computational efficiency versus accuracy is still a major problem, especially in real-time computing where response time must be less than one second. Although cross-encoders are more accurate, they can be scaled only quadratically with the size of the candidate set, and their use in the production setting is constrained. The other important problem is the domain adaptation as the majority of the existing approaches require a substantial amount of fine-tuning to succeed in the specialized domains. The lack of transfer learning mechanism across domains makes the reranking systems in practice constrained to a limited application. This weakness is especially highlighted on low-resource areas where the availability of training data is scarce. The methodologies of evaluation remain a challenge due to the fact that the normal metrics used perhaps cannot be that effective in reflecting the subtle needs of the modern systems in QA. The elaboration of newer and improved evaluation systems whereby the answers are evaluated with regard to completeness of the answers, factuality and the contextual applicability is a very important research direction. Methodology This methodology deals with reranking answers in question answering (QA) systems. Our solution is to have a two-step pipeline as the candidate answers are produced, and then reranked to enhance the accuracy. In the following, we detail the baseline for comparison, our model selection rationale, data preparation, feature engineering, mathematical formulation, architecture, and implementation specifics. Baseline method To benchmark our approach, we adopt a standard retrieve-then-read pipeline as the baseline, which combines BM25 or dense passage retrieval (DPR) with a single BERT-based reader model27. In this baseline pipeline, a question is passed to the retriever to fetch candidate answer passages, which are then fed directly to a cross-encoder reader that scores each candidate independently. The baseline does not perform ensemble-based reranking of answers, candidate answers are ranked solely by the single reader’s confidence score. This often leads to suboptimal final answers when the top-ranked passage is not truly the best source, as a single model cannot capture the full diversity of linguistic patterns across different question types. In contrast, our proposed pipeline inserts an answer reranking module that employs adaptive instance-aware weighting rather than static score fusion or knowledge compression into a single model. Model selection We proposed ensemble architecture that integrates three complementary components to achieve robust question-answer reranking. Dual-encoder layer We adopt Sentence-BERT (SBERT-base)28 to embed questions and answer sentences independently. Dense vectors are compared by cosine similarity; the top-K (= 20) candidates are passed downstream. SBERT offers millisecond-level retrieval while preserving semantic recall. Cross-encoder ensemble Three fine-tuned transformers are BERT-base16, RoBERTa-base17 and DeBERTa-base18 form the deep interaction layer. Each cross-encoder receives the concatenated [CLS] question [SEP] answer [SEP] sequence and outputs a scalar relevance score. Model diversity (different pre-training corpora and attention factorisations) guards against over-fitting to any single linguistic style. Adaptive confidence aggregator Inspired by mixture-of-experts9, we train a two-layer multilayer perceptron (MLP) that ingests the three cross-encoder scores and a one-hot vector for question class (factoid, list, yes/no, descriptive). Question types are determined using a lightweight rule-based classifier that analyses interrogative words and syntactic patterns (e.g., “how many” → list; “is/are/does” → yes/no). On a manually labelled subset of 500 questions, this classifier achieves 91.3% accuracy. A softmax layer returns weights that are applied to the raw scores before a final sigmoid. Data collection To comprehensively evaluate our methodology, three widely-adopted datasets were selected: MS MARCO v2.1 Provided by Microsoft34, MS MARCO contains approximately one million real user-generated questions paired with candidate answer passages from web documents. The dataset is devoted to open-domain QA cases, which offer linguistic diversity. Our model can easily generalize because it is a vast size (8.8 million passages). It is obtained directly out of the Bing search logs and for that reason, it is perfect in validating pragmatic reranking capacities owing to the existence of various plausible solutions to a query. WikiQA This dataset was introduced by Yang et al.35, having 3,047 natural questions based on Bing search queries, and 29,258 candidate answers sentences of Wikipedia articles. The main advantage of WikiQA is the realistic questions and the short and situation-specific responses that render it quite suitable in investigating the accuracy of reranking in simple yet noisy environments. TREC-QA TREC-QA is based on the Text Retrieval Conference QA track, and it consists of more than 1,200 factoid questions with short-answer sentences taken out of news articles36. This dataset is controlled and its answers are short in nature which would enable us to test the effectiveness of our model in identifying accurate factual information within short textual conditions. All of these datasets reflect various QA situations, which also allow us to fully validate our reranking strategy. Data preprocessing For each dataset, we followed tailored preprocessing procedures to ensure data readiness for transformer-based embedding and ranking tasks. For MS MARCO, we used the official dev set as our test set and held out 15% of the training set for validation, consistent with standard practice in the literature. For WikiQA and TREC-QA, we used the official train/dev/test splits as provided by the dataset authors (Yang et al.,35; Voorhees & Tice,36). All hyperparameters were tuned exclusively on the validation set, and test results were reported strictly on unseen test data. MS MARCO Raw data required normalization by lowercasing all text, removing redundant spaces, and tokenizing text using WordPiece tokenizer . Empty passages were discarded. No augmentation was performed given the dataset’s inherent variability. WikiQA Text normalization similar to MS MARCO was applied, with additional steps to remove irrelevant symbols and special characters. Answers with missing labels or ambiguous content were dropped. Minor data augmentation involved shuffling negative candidate answers during training to reduce bias towards positional features. TREC-QA Similar normalization and cleaning steps were conducted. Given its limited data size, additional data augmentation included question paraphrasing using back-translation techniques, thus increasing the robustness of the reranking model. Table 1 details Preprocessing pipelines for different datasets. Table 1Preprocessing pipelines for different datasets.DatasetMissing handlingTokenizationNormalizationAugmentationMS MARCORemove empty instancesWordPieceLowercase, remove spacesNoneWikiQARemove ambiguous labelsWordPieceLowercase, symbol removalNegative sample shufflingTREC-QADrop incomplete answersWordPieceLowercase, clean punctuationQuestion paraphrasingThis table outlines the preprocessing pipelines applied to MS MARCO, WikiQA, and TREC-QA datasets, detailing missing value handling, tokenization strategy, normalization techniques, and data augmentation methods employed for each dataset. Preprocessing pipelines for different datasets. This table outlines the preprocessing pipelines applied to MS MARCO, WikiQA, and TREC-QA datasets, detailing missing value handling, tokenization strategy, normalization techniques, and data augmentation methods employed for each dataset. Feature selection In developing the reranking model, we also investigated which features of the question-answer pairs are most predictive of correct answers. We employed SHAP (SHapley Additive exPlanations) analysis as a post-hoc explainability tool to interpret which input characteristics correlate with the ensemble’s ranking behavior. Note that these features are not direct inputs to the neural reranker but serve as explainability proxies computed via a surrogate model. The model inherently uses a deep representation, but for explainability we considered several indicative features derived from each question-candidate pair: e.g., semantic similarity between question and answer text, the number of named entities in the answer that also appear in the question, answer length, the original QA model’s confidence score for that candidate, etc. We used a gradient-boosted tree surrogate model (XGBoost with 500 estimators) to approximate the cross-encoder ensemble’s ranking decisions and computed SHAP values for features. The surrogate model achieves R of 0.87 on MS MARCO, 0.91 on WikiQA, and 0.89 on TREC-QA, indicating high fidelity to the actual reranker’s behavior. This analysis validates the aggregator’s input design: the two highest SHAP-ranked features across all three datasets semantic similarity (cosine score from SBERT) and named-entity overlap are directly captured by the DeBERTa and BERT cross-encoders respectively (see Table 2), providing post-hoc empirical justification for including these complementary architectures in the ensemble. The SHAP analysis serves as a post-hoc explainability tool and is not a component of the reranking pipeline itself. The following feature set is consistently computed across all three datasets: semantic similarity (cosine score from SBERT), named-entity overlap, answer length, BM25 score, and cross-encoder confidence. Table 2 reports the mean absolute SHAP values normalised to sum to 1.0 across features, computed over the full test set of each dataset. Table 2Normalized SHAP feature importance from the surrogate model across datasets (consistent feature set used for all three datasets).FeatureMS MARCOWikiQATREC-QASemantic similarity (Q-A)0.320.350.29Named-entity overlap0.230.200.24Answer length0.170.150.20This table compares normalized weighted importance scores of key features including semantic similarity, named-entity overlap, and answer length across MS MARCO, WikiQA, and TREC-QA datasets, where higher values indicate greater relevance to answer reranking. Normalized SHAP feature importance from the surrogate model across datasets (consistent feature set used for all three datasets). This table compares normalized weighted importance scores of key features including semantic similarity, named-entity overlap, and answer length across MS MARCO, WikiQA, and TREC-QA datasets, where higher values indicate greater relevance to answer reranking. Mathematical model Let Q denote a question and the top-K answer candidates retrieved by the dual encoder. The m-th cross-encoder outputs a raw relevance score . Weighted fusion. The aggregator learns weights satisfying . The fused score for candidate is Training objective. For each question we have gold label . We minimise the cross-entropy over softmax-normalised scores: We additionally regularise weight entropy to discourage extreme peaking: The final objective is . Equations (1–3) are referenced in § IV-B to discuss convergence. Model architecture Our proposed ensemble architecture integrates three complementary components to achieve robust question-answer reranking as shown in Fig. 1. Proposed ensemble model architecture for answer reranking. This figure illustrates the proposed ensemble model architecture integrating dual-encoder retrieval using SBERT and FAISS, cross-encoders including BERT, RoBERTa, and DeBERTa, adaptive aggregation, and score fusion for ranked answer prediction. Stage 1: dual-encoder retrieval Sentence-BERT (SBERT-base)28 independently encodes the question Q and each corpus sentence into 768-dimensional dense vectors. Candidate answers are ranked by cosine similarity, and the top-K () candidates are passed downstream. Approximate nearest-neighbour indexing via FAISS2 enables sub-30 ms retrieval over corpora of 5M+ sentences. Stage 2: parallel cross-encoding Each candidate pair is fed independently into three fine-tuned cross-encoders: BERT-base16, RoBERTa-base17, and DeBERTa-base18. Each model processes the concatenated [CLS] question [SEP] answer [SEP] sequence and produces a final [CLS] representation, which is passed through a task-specific linear head to obtain a scalar relevance score for the m-th encoder. Stage 3: question type classification A lightweight rule-based classifier analyses interrogative words and syntactic patterns of Q to assign a question type label from {factoid, list, yes/no, descriptive}. The classification rules map interrogative patterns as follows: “who/what/when/where” → factoid, “how many/list” → list, “is/are/does” → yes/no, and remaining → descriptive. To validate the classifier, 500 questions were randomly sampled from the MS MARCO training set and manually labeled by two annotators (Cohen’s kappa = 0.88), yielding 91.3% rule-based classification accuracy. For comparison, a learned alternative BERT-base fine-tuned on the same 500 annotations via 5-fold cross-validation achieves 93.1% accuracy, a marginal 1.8% difference. When substituted into the full pipeline on MS MARCO, the learned classifier yields MAP of 0.4021 vs. 0.4012 for the rule-based system, a 0.2% gap within one standard deviation across seeds. Given this negligible performance difference and the rule-based classifier’s practical advantages no additional training data, no GPU dependency at classification time, and full interpretability of decision rules, we retain the rule-based approach as the default. This label is encoded as a one-hot vector and concatenated with the cross-encoder scores for the aggregation stage. Stage 4: adaptive aggregation The score vector and the question-type one-hot are concatenated to form a 7-dimensional input vector and fed into a two-layer MLP (hidden size = 64, GELU activation, dropout = 0.1). A softmax output layer produces instance-aware weights satisfying . The fused score is computed via Eq. (1), followed by a sigmoid to yield a final relevance probability. Candidates are sorted in descending order of this probability. The entropy regularisation term (Eq. (3)) prevents weight collapse during training. Stage 5: output The highest-ranked answer is returned to the user. A configurable confidence threshold can trigger a “no-answer” fallback when the top-ranked candidate’s probability falls below the threshold. Because aggregation is instance-aware, the system adapts its weighting per question type. On the MS MARCO validation set, the mean learned weights are (0.42, 0.31, 0.27) for factoid questions, (0.28, 0.33, 0.39) for descriptive questions, and (0.35, 0.38, 0.27) for yes/no questions. This confirms that the system leans on BERT for short factoid answers while allocating more weight to DeBERTa for verbose descriptive answers, consistent with DeBERTa’s strength in modelling long-range dependencies through disentangled attention. Implementation details Implementation details are given below: Frameworks and libraries All experiments were conducted in PyTorch 2.2 with Hugging Face Transformers 4.41 for model fine-tuning, FAISS 1.8 for dense-vector indexing, scikit-learn 1.5 for evaluation utilities, and SHAP 0.45 for post-hoc feature attribution. Mixed-precision (FP16) training was enabled throughout. Runs were executed on a single NVIDIA A100 (40 GB) GPU, seeds were fixed to 42 and deterministic cuDNN paths forced for exact reproducibility. For all experiments, hyperparameter tuning was performed solely on the validation split, and final results were computed on the held-out test split to prevent data leakage. General training parameters General training parameters for our proposed ensemble architecture is: Dual-encoder (SBERT): one epoch of contrastive fine-tuning; learning-rate ; batch 128 Q–A pairs; weight-decay 0.01. Cross-encoders: three models (BERT, RoBERTa, DeBERTa) each trained three epochs; learning-rates , , and respectively; batch 24 pairs (grad-accumulated to 48); label-smoothing 0.1; gradient-clip 1.0. Aggregator MLP: hidden 64, GELU, dropout 0.1; five epochs at learning-rate with an entropy penalty λ = 0.02 to keep ensemble weights diverse. Training follows a sequential procedure: (1) the dual-encoder (SBERT) is fine-tuned first, (2) cross-encoders are fine-tuned independently on retrieved candidates, and (3) the aggregator MLP is trained with frozen cross-encoder weights to avoid gradient instability. Optimization techniques Optimisation is via AdamW (, , ). Time complexity For corpus size N, embedding dimension d = 768, candidate pool K = 20, sequence length L ≈ 128 tokens, and M = 3 cross-encoders, the per-question latency is Experiments and results The experimental validation of our proposed answer reranking methodology was conducted through comprehensive evaluation across multiple benchmark datasets. Our experiments include the ablation case to explain the architectural decision, quantification to compare our model with the state of the art approaches and play by play of the performance metrics that surrounds the computational efficiency. Ablation studies We performed systematic ablation studies to test the contributions of the individual components presented in our proposed architecture by ablating, or randomly altering, important components. These experiments provide empirical support for our design decisions and demonstrate additive benefits of every component through comprehensive ranking evaluation metrics across all benchmark datasets. We conducted the ablation experiments on all three datasets to have a full assessment and verify the effectiveness of the component in a variety of question-answering situations. Individual components were tested in isolation while keeping remaining architectural elements fixed, ensuring fair comparison across standard information retrieval metrics. The result of the ablation indicates that each of the components is a significant contributor to the overall performance of both ranking evaluation metrics as indicated in all the three datasets in Table 3. When the adaptive confidence aggregator is removed on MS MARCO, it causes a 3.9% drop in MAP and a 3.8% drop in MRR whereas the cross-encoder ensemble experiences 8.5% drop in MAP and 8.2% drop in MRR. Most importantly, in the absence of dual-encoder retrieval, the performance decreases substantially (0.3473 MAP, 0.3487 MRR). Table 3Impact on performance due to removal of individual components (mean over three seeds).DatasetConfigurationMAPMRRMS MARCOFull model (all components)0.40120.3970w/o Question-type encoding0.39310.3895w/o Adaptive confidence aggregator0.38540.3821w/o Cross-encoder ensemble0.36710.3643w/o Dual-encoder retrieval0.34730.3487WikiQAFull model (all components)0.87310.890w/o Question-type encoding0.86680.8652w/o Adaptive confidence aggregator0.85620.8741w/o Cross-encoder ensemble0.83510.8542w/o Dual-encoder retrieval0.84480.8605TREC-QAFull model (all components)0.9240.979w/o Question-type encoding0.9180.972w/o Adaptive confidence aggregator0.90740.9681w/o Cross-encoder ensemble0.88560.9512w/o Dual-encoder retrieval0.9130.966This table presents the ablation study results reporting MAP and MRR scores across MS MARCO, WikiQA, and TREC-QA datasets upon sequential removal of the adaptive confidence aggregator, cross-encoder ensemble, and dual-encoder retrieval components from the full model. Impact on performance due to removal of individual components (mean over three seeds). This table presents the ablation study results reporting MAP and MRR scores across MS MARCO, WikiQA, and TREC-QA datasets upon sequential removal of the adaptive confidence aggregator, cross-encoder ensemble, and dual-encoder retrieval components from the full model. On WikiQA, the component contributions follow a similar pattern but with smaller absolute differences due to the dataset’s higher baseline performance. The adaptive confidence aggregator provides 1.9% MAP improvement (0.8562 to 0.8731) and 1.8% MRR enhancement (0.8741 to 0.890). The cross-encoder ensemble contributes 4.5% MAP gain (0.8351 to 0.8731) and 4.1% MRR improvement (0.8542 to 0.890). Without dual-encoder retrieval, performance drops to 0.8448 MAP and 0.8605 MRR. TREC-QA results show consistent component importance with 1.8% MAP improvement from adaptive confidence aggregator (0.9074 to 0.924) and 1.1% MRR gain (0.9681 to 0.979). The cross-encoder ensemble provides 4.3% MAP enhancement (0.8856 to 0.924) and 2.9% MRR improvement (0.9512 to 0.979). Removing dual-encoder retrieval reduces performance to 0.913 MAP and 0.966 MRR. To further analyze robustness, we present a question-type performance breakdown on the MS MARCO validation set. Our model achieves MAP of 0.43 (factoid), 0.38 (descriptive), 0.41 (yes/no), and 0.37 (list). The adaptive weighting shows the largest gains for descriptive questions (+4.1% over uniform weighting) and smallest for factoid questions (+1.8%), consistent with the learned weight distributions reported in Section "Stage 4: adaptive aggregation", where DeBERTa receives higher weight (0.39) for descriptive questions and BERT dominates (0.42) for factoid questions. Failure cases are primarily observed for ambiguous list-type questions where the rule-based classifier misidentifies question type, suggesting that improving the question classifier could yield further gains. Additionally, the ablation removing question-type encoding (Table 3, row ’w/o question-type encoding’) confirms a 2.0% MAP drop on MS MARCO, validating that the 4-dimensional type vector provides a measurable instance-aware contribution independent of the ensemble itself. Quantitative failure analysis on the MS MARCO validation set (error rate = 1 - P@1) shows: factoid 14.8%, yes/no 19.1%, descriptive 24.7%, and list 31.2%. The highest list-type error rate coincides with the classifier’s highest misclassification rate (14.3% for list vs. 4.1% for descriptive), confirming that ambiguous list questions are the primary failure mode. On WikiQA, multi-sentence answer candidates produce 18.3% higher error rates than single-sentence candidates, indicating that answer granularity is a secondary failure factor independent of question-type classification. Additionally, the ablation removing question-type encoding confirms a 2.0% MAP drop on MS MARCO, validating that the 4-dimensional type vector provides a measurable instance-aware contribution independent of the ensemble itself. Quantitative analysis We evaluated our proposed methodology against established state-of-the-art approaches across three benchmark datasets using standard information retrieval ranking metrics. It compares recent transformer-based models and the standard learning-to-rank methods and gives full coverage of the existing approaches with prioritized emphasis on ranking-conscious evaluation. We further contextualize our results against recent strong baselines from different reranking paradigms. ColBERTv28 is a late-interaction retrieval model that achieves state-of-the-art MRR@10 as a standalone retriever on the MS MARCO passage ranking task. Our model achieves MRR of 0.3970 on the same benchmark. It is important to note that ColBERTv2 operates as a full-corpus retriever whereas our framework is a reranker applied to pre-retrieved top-K candidates—the evaluation settings are not directly equivalent, and a controlled comparison under identical candidate sets would require re-running ColBERTv2 in a reranking-only configuration, which is beyond the scope of the current study.11 (GPT-4 based listwise reranker) incurs multi-second per-query latency versus our 54.9 ms with parallel cross-encoder execution, making the latency trade-off a primary distinguishing factor. Neither ColBERTv2 nor RankGPT reports results on WikiQA or TREC-QA; a cross-benchmark comparison under identical settings remains an open direction for future work. To ensure statistical robustness, all experiments were repeated with three random seeds (42, 123, 456), and results in Tables 4 and 5 report the mean and standard deviation across these runs. Statistical significance was assessed using a two-tailed paired t-test on per-query MAP scores across the full test sets: 6,980 queries (MS MARCO dev set), 1,040 queries (WikiQA test set), and 284 queries (TREC-QA test set). For each baseline comparison, per-query MAP scores were paired across three random seeds (42, 123, 456). All reported improvements are statistically significant at p < 0.05; the smallest observed test statistic was t(283) = 3.41 (p = 0.0007) for the TREC-QA comparison against BERT-BiG. The MS MARCO query count of 6,980 is consistent with the standard MS MARCO dev subset used in the literature8. Table 4Comprehensive evaluation results on MS MARCO, WikiQA, and TREC-QA datasets.DatasetModelMAPMRRMS MARCOBM25+BERT(Shallow)370.25940.2277monoT5-base30.36210.3658BERT-LargeBM25CAT200.3670.369RankT5150.37100.3742Proposed (ours)0.4012±0.0030.3970±0.004WikiQACKANN130.7320.755TANDA40.8380.852BERT-BiG300.8550.863Proposed0.8731±0.0050.890±0.004TREC-QAaNMM220.74950.8109CKANN130.8190.863BERT-BiG300.9130.966Proposed0.924±0.0020.979±0.002This table presents a comprehensive comparison of the proposed model against state-of-the-art baselines on MS MARCO, WikiQA, and TREC-QA datasets, reporting MAP and MRR scores with mean and standard deviation over three seeds. † indicates that the results are reported as the mean and standard deviation over three seeds, as the proposed model’s scores are presented with this statistical measure (e.g., 0.8731±0.005) to ensure reliability and reproducibility of the results. Comprehensive evaluation results on MS MARCO, WikiQA, and TREC-QA datasets. This table presents a comprehensive comparison of the proposed model against state-of-the-art baselines on MS MARCO, WikiQA, and TREC-QA datasets, reporting MAP and MRR scores with mean and standard deviation over three seeds. † indicates that the results are reported as the mean and standard deviation over three seeds, as the proposed model’s scores are presented with this statistical measure (e.g., 0.8731±0.005) to ensure reliability and reproducibility of the results. Table 5Additional ranking metrics (mean ± std over three seeds).DatasetNDCG@10P@1P@5MS MARCO0.4458±0.0040.4215±0.0050.3187±0.003WikiQA0.8946±0.0030.9012±0.0060.7824±0.004TREC-QA0.9417±0.0020.9583±0.0030.8950±0.002This table reports additional ranking metrics including NDCG@10, P@1, and P@5 with mean and standard deviation over three seeds across MS MARCO, WikiQA, and TREC-QA datasets, further validating the proposed model’s ranking effectiveness. Additional ranking metrics (mean ± std over three seeds). This table reports additional ranking metrics including NDCG@10, P@1, and P@5 with mean and standard deviation over three seeds across MS MARCO, WikiQA, and TREC-QA datasets, further validating the proposed model’s ranking effectiveness. Our proposed model consistently outperforms existing approaches across all three datasets and evaluation metrics as shown in Table 4, with all improvements statistically significant (p < 0.05, paired t-test over three seeds). Table 5 reports additional ranking metrics including NDCG@10, P@1, and P@5. On MS MARCO, we achieve an 8.1% improvement in MAP (0.4012 vs 0.371) over the recent RankT5 baseline and 9.3% over BERT-LargeBM25CAT, demonstrating significant advancement in ranking quality. This represents substantial progress on a challenging large-scale dataset where incremental improvements are typically difficult to achieve. The results of the WikiQA show that MAP (0.8731 vs 0.855) and MRR (0.890 vs 0.863) improve by 2.1% and 3.1% respectively over the BERT-BiG baseline which is a significant improvement given the maturity of the data and the competitiveness of the current methods. In the area of MRR, our model shows a very good performance, which implies that it is more likely to put correct answers in high positions and this is supported by the P@1 value of 0.9012 (Table 5). On TREC-QA, we achieve 0.924 MAP and 0.979 MRR, representing 1.2% and 1.3% improvement over the strongest compared baseline BERT-BiG. MRR of nearly perfect score 0.979 and NDCG@10 of 0.9417 indicate the outstanding ability of ranking correct answers on the first position of ranking lists, which is important in terms of user satisfaction of question-answering systems. The performance of three performance question-answering systems (MS MARCO, WikiQA, and TREC-QA) with varying candidate answers (K), varying between 5 to 40, is shown in Fig. 2. TREC-QA performs best (0.89-0.92 MAP), then the next best performance is WikiQA (0.82-0.86 MAP), and lastly, the lowest performance is MS MARCO (0.32-0.39 MAP). The red dashed vertical at K = 20 line represents an ideal operating point and where most systems begin increasing their performance but with a manageable, computational cost. MAP vs. candidate pool size (K) across all datasets. This figure presents the Mean Average Precision (MAP) comparison across MS MARCO, WikiQA, and TREC-QA datasets as the candidate pool size increases, demonstrating optimal performance achieved at K=20. Figure 3 presents a bar chart that compares the performance of a question-answering system across three datasets (MS MARCO, WikiQA, and TREC-QA) using two evaluation metrics: MAP (mean average precision) in blue and MRR (mean reciprocal rank) in red. The system performs poorly on MS MARCO ( 0.40 for both metrics), moderately well on WikiQA ( 0.87-0.88), and achieves the best performance on TREC-QA ( 0.93-0.98). The consistently higher MRR scores compared to MAP scores across all datasets suggests that correct answers are typically ranked high, but there may be fewer relevant answers per query overall. MAP and MRR comparison across datasets. This figure shows contrasts performance measure of MAP and MRR results of the proposed model against the MS MARCO, WikiQA, and TREC-QA benchmark datasets, showing that the model has prevailed in all the test conditions. Figure 4 provides SHAP (SHapley Additive exPlanations) feature impact analysis of the reranking scores of answers produced by different features in the MS MARCO dataset. Semantic Match Score produces the greatest positive effect on ranking scores, and the majority of them have high positive values of SHAP (red points on the right). The answer length ratio, and BM25 score also have a positive but less reliable effect, whereas such features as the position of an answer are generally negatively impacting (blue points on the left) which implies that shorter answers or those that are found earlier in passages are rated lower. SHAP feature impact analysis on MS MARCO. This is a figure of SHAP-based feature impact analysis on the MS MARCO answer reranking task to determine the most important features that affect the ranking decisions by the model. Figure 5 depicts SHAP (SHapley Additive exPlanations) feature impact analysis that indicates the impact of various features on the scores of answers selection in the WIKIQA dataset. Semantic match score produces the greatest positive effect on ranking scores, and the majority of them have high positive values of SHAP (red points on the right). Word overlap count and BM25 score also have a positive contribution with a significant difference whereas sentence length and the question answer interaction feature exhibit both positive and negative contribution around zero. The features of lower rank such as entity match ratio, answer position, syntactic similarity, topic coherence and stopword count have little impact meaning that they have a relatively lower impact on the final reranking decisions. SHAP feature impact analysis on WikiQA. This figure presents SHAP-based feature importance analysis with answer-reranking task in the WikiQA, in a way that shows the relative importance of each feature to the score that the model makes in prediction. Figure 6 provides SHAP analysis of TREC-QA, which indicates the role of various features on scores of selecting answers. The most significant features are Semantic Similarity (BERT) and BM25 score, which have the strongest positive changes to the ranking scores with dense red points on the right. There are also positive contributions by lexical overlap, query answer token interaction, and named entity match ratio with a relatively small magnitude. Conversely, answer position/rank (red points on the left) is significantly affected (mainly negatively), meaning that the position of the answer or its initial rank can be punished during reranking whereas attributes like answer length, syntactic similarity and stopword ratio have a relatively weak effect on ultimate ranking decisions. SHAP feature impact analysis on TREC-QA. This figure shows SHAP-based feature importance analysis of the TREC-QA answer reranking task, which shows the important features that the ranking performance of the model varies with the candidate answer. Performance analysis In addition to metrics of accuracy, we performed an all-inclusive assessment of the quality of computational efficiency and scalability. The efficiency of performance analysis includes the efficiency of training and the latency of inference. Overall training complexity For the complete ensemble-based answer reranking system with specified parameters: Where: E = number of training epochs E = number of training epochs N = total number of training questions N = total number of training questions L ≈ 128 tokens (sequence length) L ≈ 128 tokens (sequence length) d = 768 (embedding dimension) d = 768 (embedding dimension) K = 20 (candidate pool size) K = 20 (candidate pool size) M = 3 (number of cross-encoders) M = 3 (number of cross-encoders) Component-wise training complexity Component-wise training complexity for an ensemble-based answer reranking system is given below. Dual-encoder training phase5Single epoch fine-tuning of SBERTComplexity: Dual-encoder training phase5Single epoch fine-tuning of SBERTComplexity: Single epoch fine-tuning of SBERT Single epoch fine-tuning of SBERT Complexity: Complexity: Cross-encoder ensemble training phase6Three cross-encoders (BERT, RoBERTa, DeBERTa)Complexity: Cross-encoder ensemble training phase6Three cross-encoders (BERT, RoBERTa, DeBERTa)Complexity: Three cross-encoders (BERT, RoBERTa, DeBERTa) Three cross-encoders (BERT, RoBERTa, DeBERTa) Complexity: Complexity: Adaptive aggregator training phase7MLP training with cross-encoder outputsComplexity: Adaptive aggregator training phase7MLP training with cross-encoder outputsComplexity: MLP training with cross-encoder outputs MLP training with cross-encoder outputs Complexity: Complexity: Total training complexity FLOPs This simplifies to: where the quadratic sequence length dominates. Inference time complexity Based on provided formula with parameters N (corpus size), d = 768, K = 20, L ≈ 128, M = 3: Component-wise inference breakdown Training complexity The complexity of ensemble-based answer reranking system when trained component-wise is as follows. Retrieval phase11FAISS approximate nearest neighbor search: Embedding computation: Practical value: Retrieval phase11FAISS approximate nearest neighbor search: Embedding computation: Practical value: FAISS approximate nearest neighbor search: FAISS approximate nearest neighbor search: Embedding computation: Embedding computation: Practical value: Practical value: Cross-encoder phase12M = 3 cross-encodersK = 20 candidatesPractical value: Cross-encoder phase12M = 3 cross-encodersK = 20 candidatesPractical value: M = 3 cross-encoders M = 3 cross-encoders K = 20 candidates K = 20 candidates Practical value: Practical value: Aggregator phase13M = 3 modelsK = 20 candidatesPractical value: Total inference complexity14 FLOPs per query Aggregator phase13M = 3 modelsK = 20 candidatesPractical value: Total inference complexity14 FLOPs per query M = 3 models M = 3 models K = 20 candidates K = 20 candidates Practical value: Practical value: Since cross-encoder processing dominates: Empirical latency measurements To complement the theoretical analysis, we measured actual wall-clock inference latencies on a single NVIDIA A100 (40 GB) GPU with batch size 1 (single query). Table 6 reports the per-query latency breakdown averaged over 1,000 queries from MS MARCO. Table 6Per-query inference latency breakdown on NVIDIA A100 GPU.ComponentLatency (ms)SBERT encoding + FAISS retrieval8.4BERT cross-encoder (K=20)42.1RoBERTa cross-encoder (K=20)43.7DeBERTa cross-encoder (K=20)46.2Adaptive aggregator MLP0.3Total (sequential)140.7Total (parallel cross-enc.)54.9This table presents the per-query inference latency breakdown of individual model components on an NVIDIA A100 GPU, comparing total sequential and parallel cross-encoder execution times in milliseconds. Per-query inference latency breakdown on NVIDIA A100 GPU. This table presents the per-query inference latency breakdown of individual model components on an NVIDIA A100 GPU, comparing total sequential and parallel cross-encoder execution times in milliseconds. When cross-encoders are executed in parallel across three GPU streams, end-to-end latency is 54.9 ms per query, which is feasible for near-real-time applications. Peak GPU memory consumption is 6.8 GB with parallel execution and FP16 inference, well within the capacity of modern inference hardware. At batch size 32, throughput reaches 185 queries/second. To quantify the cost-vs-gain trade-off, a single BERT cross-encoder achieves 0.3671 MAP on MS MARCO with approximately 50.5 ms latency (SBERT/FAISS retrieval 8.4 ms + BERT cross-encoder 42.1 ms), while our full ensemble achieves 0.4012 MAP with 54.9 ms latency under parallel execution a 9.3% accuracy gain for only 8.7% additional latency. The peak GPU memory of 6.8 GB (FP16) fits within a single consumer-grade GPU such as the NVIDIA RTX 3090, where our measured latency is 71.3 ms per query 30% overhead vs. the A100 but within interactive response thresholds. For CPU-only deployment (Intel Xeon E5-2670, 16-core), sequential cross-encoder execution requires approximately 1,840 ms per query; applying INT8 quantization via Hugging Face Optimum reduces this to approximately 620 ms, suitable for batch or asynchronous QA pipelines. A single-encoder fallback (DeBERTa-only, no ensemble) further reduces CPU latency to approximately 310 ms at a cost of 2.4% MAP on MS MARCO. For scalability, increasing the candidate pool from K=20 to K=40 approximately doubles cross-encoder latency to 10 ms on GPU, remaining within interactive response times. Entropy regularisation sensitivity We evaluated entropy regularisation parameter across all three datasets. On MS MARCO: λ = 0 causes weight collapse to a single model (MAP 0.3891); performance peaks at λ = 0.02 (MAP 0.4012); over-regularisation at λ = 0.1 forces near-uniform weights (MAP 0.3934). On WikiQA: the same pattern holds—peak MAP at λ = 0.02 (MAP 0.8731), degrading to 0.8684 at λ = 0.1 and 0.8663 at λ = 0. On TREC-QA: peak MAP occurs at λ = 0.01 and λ = 0.02 (MAP 0.925 and 0.924 respectively), confirming that λ = 0.02 is robust across all three datasets. For new domains, we recommend tuning λ on the validation set within the range . Through our overall experimental analysis, it is demonstrated that the proposed ensemble-based answer reranking method achieves state-of-the-art performance with computational efficiency acceptable for real-world deployment. The systematic ablation experiments confirm the contribution of every architectural element, whereas quantitative comparisons demonstrate consistent improvements over current techniques on a variety of benchmark datasets. Discussion The experimental results show that ensemble-based answer reranking is effective across the three benchmark datasets evaluated, spanning open-domain web questions (MS MARCO), Wikipedia-based QA (WikiQA), and factoid QA from news (TREC-QA). Generalization to other domains such as biomedical or legal QA requires further validation. The proposed methodology achieves 9.3% MAP and 7.6% MRR improvement on MS MARCO over BERTLargeBM25CAT, indicating meaningful progress in ranking quality among the compared baselines. This is because the combination of dual-encoder retrieval, cross-encoder ensemble processing, and adaptive confidence aggregation do so synergistically and overcome the shortcomings of single-model methods. The ablation experiments demonstrate the important information about the component contributions. Elimination of the adaptive confidence aggregator leads to overall performance loss (3.9% MAP loss on MS MARCO, 1.9% on WikiQA, 1.8% on TREC-QA) that shows the relevance of instance-based model weighting. More importantly, the cross-encoder ensemble provides the most significant performance improvement (8.5% MAP increase on MS MARCO) as it confirms the hypothesis that diversity of models improves the accuracy of answer selection by complementary linguistic pattern recognition. These differences in the performances of the datasets are interesting displays of the nature of the data they represent. The enhancements of ensemble methods on the difficult large-scale web-derived data are the largest on MS MARCO, which indicates that the different model attitudes would be especially useful to noisy real-world data. The smaller improvement on TREC-QA (1.2% MAP) is explained by its controlled factoid structure, where strong single-model baselines already achieve high precision, leaving less headroom for ensemble gains. The improvement in intermediate performance (2.1% MAP) of WikiQA can be attributed to the fact that it is neither too complex nor too simple in its approach to semantic understanding because the ensemble diversity is useful and the computation overhead can be controlled. The greater difference in MRR scores between the MAP scores in all the data sets prove that the proposed system can be successfully used to place correct answers at the top positions that are essential in the context of practical implementation where a user is likely to focus mainly on the initial results. However, despite the strong performance, several limitations should be acknowledged. The quadratic complexity in cross-encoder processing constrains scalability for extremely large candidate sets. With parallel GPU execution, the end-to-end latency of 54.9 ms per query (Table 6) is feasible for near-real-time applications but may be prohibitive for latency-critical deployments requiring sub-10 ms response times. Additionally, the rule-based question classifier may misclassify ambiguous question types; the evaluation is limited to English-language QA benchmarks; the framework has not been tested in domain-specific settings such as biomedical or legal QA; and the full three-encoder ensemble requires a modern GPU for sub-100 ms latency CPU-only deployment increases end-to-end latency to approximately 620 ms with INT8 quantization, and a single-encoder DeBERTa fallback is recommended for strict resource-constrained settings. Further limitations include: the framework assumes pre-retrieved candidate sets and does not address end-to-end retrieval quality, and the ensemble approach requires fine-tuning three separate transformer models, increasing training time and storage requirements compared to single-model approaches. The SHAP surrogate model (R = 0.87–0.91) confirms that the features most predictive of correct ranking align with the encoder strengths exploited by the adaptive aggregator: BERT’s higher mean weight (0.42) for factoid questions corresponds to its strength on entity-anchored retrieval, which is reflected in named-entity overlap being the second-highest SHAP feature (Table 2); DeBERTa’s higher mean weight (0.39) for descriptive questions corresponds to the greater importance of answer length and semantic coherence signals in those question types, as observed in the SHAP analysis across all datasets. Conclusion and future recommendations The study has been able to show that answering questions by using ensemble-based answer reranking with adaptive confidence weighting can greatly improve the quality of performance of question answering but is also computationally feasible. The suggested methodology has shown impressive gains in both benchmark datasets, as 9.3% improvement in MAP on MS MARCO and almost perfect 0.979 MRR on TREC-QA demonstrate new records of improvement. The systematized ablation experiments validate that every architectural component is beneficial to the overall system performance, and the cross-encoder ensemble offers the largest benefits, and the adaptive confidence aggregator makes sure that the model is fully utilized. The mathematical model which includes entropy regularization appropriately helps to avoid the over-concentration of models and allows the dynamic weight adjustment to the properties of questions. These results confirm the hypothesis that intelligent ensemble integration can be used to overcome the constraints of single-model methods in a non-prohibitive manner and provide high-end reranking to be practically implemented in large-scale question answering systems. Despite these gains, the framework has limitations including quadratic cross-encoder complexity for large candidate sets, dependence on a rule-based question classifier, and evaluation restricted to English-language benchmarks. Future studies need to investigate adaptive ensemble pruning to save on computational expenses whilst keeping the accuracy intact and incorporate multimodal information source on complex queries. Acknowledgements This work was supported by the institutional research project Digital Twins and Smart Solutions at the Faculty of Organization and Informatics, University of Zagreb, funded by the European Union – NextGenerationEU. This research was also supported by Princess Nourah bint Abdulrahman University Researchers Supporting Project Number (PNURSP2026R97), Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia. Author contributions T. Kehkashan, and M. Abdelhaq designed the research, proposed the method, and drafted the main manuscript. S. S. S. Ahmad, N. Z. Ismail contributed to predictive model development and explanation of the results. A. Akhunzada carried out technical supervision, refined the manuscript, and supervised general research work. N. Ivkovi´c contributed to the literature review and data curation. M. Abdullah carried out data preprocessing, experiment setup, and results visualization with supervision. All authors read and approved the final version of the manuscript. Funding This work was supported by the institutional research project Digital Twins and Smart Solutions at the Faculty of Organization and Informatics, University of Zagreb, funded by the European Union – NextGenerationEU. This research was also supported by Princess Nourah bint Abdulrahman University Researchers Supporting Project Number (PNURSP2026R97), Princess Nourah bint Abdulrahman University, Riyadh, Saudi Arabia. We also appreciate technical insights and collaborative support provided by the VLCMatrix Lab during this research. Data availability The datasets used in this study are publicly available. The MS MARCO dataset, comprising over one million real Bing queries for question answering and passage ranking, is accessible via the official website at https://microsoft.github.io/msmarco/, with all dataset download links available at https://microsoft.github.io/msmarco/Datasets.html; this dataset is made available for non-commercial research use only, and users must accept the terms on the official page to access download links. The WikiQA dataset, containing 3,047 questions and 29,258 sentence pairs for opendomain question answering, can be downloaded from the official Microsoft page at https://www.microsoft.com/en-us/download/details.aspx?id=52419 and is also accessible via Hugging Face at https://huggingface.co/datasets/microsoft/wiki_qa; it is freely available for research use only. The TREC-QA dataset, a standard benchmark for factoid question answering from the NIST TREC Question Answering track, is available through the official NIST TREC QA page at https://trec.nist.gov/data/qa.html, and a cleaned version is additionally accessible via GitHub at https://github.com/sid7954/TrecQA. Declarations Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Tanzila Kehkashan and Muhammad Abdullah: These authors contributed equally to this work. Contributor Information Muhammad Abdullah, Email: malikabdmalik468@gmail.com. Nikola Ivković, Email: nikola.ivkovic@foi.hr. References Associated Data Data Availability Statement The datasets used in this study are publicly available. The MS MARCO dataset, comprising over one million real Bing queries for question answering and passage ranking, is accessible via the official website at https://microsoft.github.io/msmarco/, with all dataset download links available at https://microsoft.github.io/msmarco/Datasets.html; this dataset is made available for non-commercial research use only, and users must accept the terms on the official page to access download links. The WikiQA dataset, containing 3,047 questions and 29,258 sentence pairs for opendomain question answering, can be downloaded from the official Microsoft page at https://www.microsoft.com/en-us/download/details.aspx?id=52419 and is also accessible via Hugging Face at https://huggingface.co/datasets/microsoft/wiki_qa; it is freely available for research use only. The TREC-QA dataset, a standard benchmark for factoid question answering from the NIST TREC Question Answering track, is available through the official NIST TREC QA page at https://trec.nist.gov/data/qa.html, and a cleaned version is additionally accessible via GitHub at https://github.com/sid7954/TrecQA.