RAPT: Retrieval-Augmented Post-hoc Thresholding for Multi-Label Classification
Lasal Jayawardena, Nirmalie Wiratunga, Ikechukwu Nkisi-Orji, Darren Nicol · 2026
Read RAPT: Retrieval-Augmented Post-hoc Thresholding for Multi-Label Classification by Lasal Jayawardena, Nirmalie Wiratunga, Ikechukwu Nkisi-Orji, Darren Nicol (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16535.txt.
Introduction
Jayawardena and three colleagues offer RAPT, a wrapper that sharpens label choice for industrial multi-label document pipelines without retraining the classifier. For each document, RAPT retrieves similar past thresholding cases and adjusts the cutoff from their outcomes. Across one industrial dataset and six public benchmarks, it beat fixed global and per-label thresholds. With metric learners it reached 0.87 Macro-F1, while transformer variants averaged 0.775 Macro-F1 in the same tests. It also beat few-shot language-model baselines with far less inference time and graphics memory.
Catalog record
| CREATOR | Lasal Jayawardena, Nirmalie Wiratunga, Ikechukwu Nkisi-Orji, Darren Nicol |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38532 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.16535
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RAPT: Retrieval-Augmented Post-hoc Thresholding for Multi-Label Classification Lasal Jayawardena, Nirmalie Wiratunga, Ikechukwu Nkisi-Orji, Darren Nicol Abstract Industrial multi-label document understanding pipelines score candidate labels and threshold or rank them to form a label set per document. This early selection step directly affects the accuracy of downstream information extraction from the document, as well as the associated verification effort. In practice, OCR noise, label imbalance, instance-dependent label cardinality, and asymmetric error costs make global score thresholds brittle and hard to maintain as document formats evolve. We present Rapt , a deployment-oriented retrieval-augmented score thresholding wrapper, applied post-hoc to improve label set selection without retraining the underlying classifier. Rapt is a model-agnostic wrapper: any predictor that provides document representations for similarity search and per label confidence scores can be used, including metric learning encoders and fine-tuned transformer classifiers. For each query document, given a classifier’s score vector, Rapt retrieves similar document thresholding situations (cases) and adapts the query’s label set selection threshold using their outcomes. The adaptation selects the final label set by locally aggregating neighbour solutions (e.g. average label count, cutoff calibration). Evaluation compared multi-label classifiers (metric learners and transformers) combined with Rapt against global and label-wise thresholding baselines, and against few-shot LLMs. Across an industrial dataset and six public benchmarks, Rapt consistently outperformed global and label-wise static thresholding baselines. In the industrial setting, Rapt achieved its best predictive performance with metric learners, reaching 0.87 Macro-F1, while fine-tuned transformer variants on average achieved 0.775 Macro-F1, outperforming few-shot LLM baselines ( K = 5 K=5 ) by 2 × 2\times and requiring at least 115 × 115\times less inference time and 13.5 × 13.5\times less GPU memory. 1 Introduction Unlike traditional single-label classification, multi-label classification requires predicting a set of labels for each instance. This capability has become central to many high-impact applications in text, vision, recommendation, and decision support [ 27 , 35 ] . The practical difficulty of multi-label classification is not only learning good label rankings, but deciding which labels to actually assign at test time when label cardinality varies across instances, and label co-occurrence patterns vary across the instance space. Typically, models output per label activations that imply a label ranking, but deployment requires a cutoff rule that maps activation scores to a variable size label set, often with unequal penalties for false positives (FPs, suprious labels) and false negatives (FNs, missed labels). Global decision rules, such as fixed score thresholds, are attractive for simplicity. Still, they can be hard to specify in skewed spaces where the semantics of a given score value can differ across regions of the instance space and across labels. In document understanding pipelines, multi-label predictions often determine pre-processing pathways. A typical workflow, shown in Fig. 1 , converts raw documents to PDF, applies OCR, performs document categorisation, identifies areas of interest, and then runs document type and class specific attribute extraction. In Fig. 1 , the predicted label set triggers downstream tasks, so selection errors can propagate through subsequent stages before human verification. Omitted labels (FNs) may prevent appropriate tasks from running, while spurious labels (FPs) can trigger inappropriate processes, yielding irrelevant or misleading fields and increasing verification effort and operational risk. This challenge is particularly evident in industrial settings, where organisations maintain large collections of legacy documents in heterogeneous formats, including scanned paper records, reports, and correspondence containing operational and customer information. In collaboration with WM Nicol, an industry partner operating in the transportation and waste management domain, we consider document processing pipelines that must handle diverse document types whose content may correspond to multiple operational categories simultaneously. Accurate prediction of the label set is therefore critical to routing each document correctly to downstream extraction, review, and case handling workflows. To address this multi-label classification challenge, we introduce Rapt , a lightweight retrieval-augmented wrapper employing case-based reasoning (CBR). It can be applied post-hoc to any multi-label classifier producing embeddings and label-wise scores, enabling instance-specific thresholding and label set selection. In addition to benchmark data, the dataset used in this study is derived from WM Nicol’s operational documents which provides a realistic setting where thresholding decisions directly influence downstream processing. The contributions of this work are outlined as follows: • propose a framework for constructing a case base from the training set, in which each case consists of a multi-label classification instance and threshold values inferred from the ground truth labels associated with its score vector, eliminating the need for additional data or human annotation; • introduce neighbourhood-driven dynamic thresholding strategies for variable cardinality set prediction, including AvgCount and Rank Calibration, and show how residual error information from retrieved cases can be used to adapt the multi-label score vector prior to thresholding; and • evaluate Rapt on an industrial document workflow dataset, and six public benchmarks showing consistent gains over global and label-wise thresholding baselines, with zero-shot and few-shot LLM comparators also reported. Figure 1: Industry use case pipeline for document processing and information extraction. Left: a typical document with areas indicated for class label assignment. Top: multi-label predictions act as routing decisions for downstream extraction tasks. Bottom: typical prediction scores from a backbone model on y-axis, shown relative to class specific optimal thresholds, with x-axis showing the label set. This illustrates how a global threshold (set at 0.5 for instance) can misclassify labels, omitting required classes (class label 3) and including spurious ones (class label 17 and 18). 2 Related Work Multi-label classification methods are commonly divided into problem transformation and algorithm adaptation approaches [ 27 , 35 ] . Methods such as Binary Relevance, Classifier Chains [ 19 ] , RAkEL [ 28 ] , and Calibrated Label Ranking [ 7 ] differ in how they model label dependence, but they still typically produce label-wise scores or rankings that must be converted into a final predicted label set. A central but often underemphasised part of this conversion is therefore thresholding: deciding where the relevance boundary should lie for a given instance. Prior work has studied global and label-specific thresholding strategies, including SCut [ 33 ] , MCut [ 14 ] , MetaLabeler [ 25 ] , MALTOR [ 12 ] , and more recent neighbourhood-aware approaches [ 20 ] . However, most methods either learn global thresholding rules or rely on additional predictive models, leaving open the question of how thresholding decisions might instead be derived directly from local evidence in similar labelled instances. Case-based reasoning solves a new problem by retrieving similar prior cases and adapting their solutions to the current query [ 1 ] . This distinguishes CBR from neighbourhood-based methods that primarily aggregate local evidence, such as ML-kNN [ 34 ] or Deep k-Nearest Neighbours [ 18 ] . In this sense, useful retrieval is not only about finding similar neighbours, but about whether their solutions can be revised for the new instance [ 22 , 5 ] . Our work adopts this adaptation perspective, but applies it to a different target: rather than using retrieved cases for direct label voting, we use them to adapt the score-to-set conversion step in multi-label prediction. Episodic metric meta-learning provides a natural backbone for our framework because it jointly supports retrieval and prediction. Matching Networks [ 30 ] condition predictions on