Calibrated Abstention for Reliable TCR–pMHC Binding Prediction under Epitope Shift

Arman Bekov, Timur Bekzhanov, Bekzat Sadykov · 2026

Read Calibrated Abstention for Reliable TCR–pMHC Binding Prediction under Epitope Shift by Arman Bekov, Timur Bekzhanov, Bekzat Sadykov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13254.txt.

Introduction

Bekov, Bekzhanov, and Sadykov frame T-cell receptor binding prediction as selective prediction, where a model either reports a sound confidence score or abstains. Their dual-encoder model reads CDR3 alpha, CDR3 beta, and peptide sequences through a ready protein language model. Temperature scaling fixes miscalibration, and a conformal abstention rule holds coverage at a chosen error rate. Under epitope-held-out testing it reaches AUROC 0.813 and calibration error 0.043, cutting error from 18.7 percent to 10.9 percent at 80 percent coverage.

Catalog record

CREATORArman Bekov, Timur Bekzhanov, Bekzat Sadykov
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19381
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13254

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Calibrated Abstention for Reliable TCR–pMHC Binding Prediction under Epitope Shift Arman Bekov, Timur Bekzhanov, Bekzat Sadykov Abstract Predicting T-cell receptor (TCR)–peptide-MHC (pMHC) binding is central to vaccine design and T-cell therapy, yet deployed models frequently encounter epitopes unseen during training, causing silent overconfidence and unreliable prioritization. We address this by framing TCR–pMHC prediction as a selective prediction problem: a calibrated model should either output a trustworthy confidence score or explicitly abstain. Concretely, we (1) introduce a dual-encoder architecture encoding both CDR3 α \alpha /CDR3 β \beta and peptide sequences via a pre-trained protein language model; (2) apply temperature scaling to correct systematic probability miscalibration; and (3) impose a conformal abstention rule that provides finite-sample coverage guarantees at a user-specified target error rate. Evaluated under three split strategies—random, epitope-held-out, and distance-aware—our method achieves AUROC 0.813 and ECE 0.043 under the challenging epitope-held-out protocol, reducing ECE by 69.7% relative to an uncalibrated baseline. At 80% coverage, the selective model further reduces error rate from 18.7% to 10.9%, demonstrating that calibrated abstention enables principled coverage–risk trade-offs aligned with practical screening budgets. I Introduction TCR–pMHC binding prediction underpins computational screening pipelines for cancer neoantigen prioritization, infectious disease response, and adoptive cell therapy design [ 19 , 4 ] . Modern sequence-based models achieve strong aggregate accuracy on held-in epitopes, but deployment invariably involves novel peptide targets whose binding landscape was not covered during training. This epitope shift is a form of covariate shift that is largely invisible to standard random-split evaluation yet can dramatically inflate false-positive rates in wet-lab follow-up [ 18 , 10 ] . Two intertwined failure modes arise under epitope shift. First, neural classifiers are miscalibrated: their raw sigmoid outputs overestimate confidence on out-of-distribution inputs, providing false assurance for screening decisions [ 5 ] . Second, because downstream validation is expensive—each candidate pair may require 10^{3} – 10^{4} USD of functional assay—the cost is asymmetric: advancing false positives wastes resources, while missing true binders is an opportunity cost. Neither failure mode is captured by AUROC or AUPRC alone. We address both issues through calibrated selective abstention : the model emits a calibrated probability if its uncertainty is below a threshold, and defers otherwise. This converts the model output into a control signal for screening budgets, enabling practitioners to explicitly trade coverage (fraction of candidates scored) for error rate. Our contributions are: 1. A dual-encoder backbone that jointly encodes TCR CDR3 sequences and peptide sequences using ESM-2 representations, with class-weighted training to handle severe label imbalance ( \sim 4% positive rate). 2. Post-hoc temperature scaling calibration fitted on a split-matched calibration set, reducing ECE by over 50% on all split types. 3. A split-conformalized abstention rule that provides a finite-sample marginal coverage guarantee: at target error rate \varepsilon , the empirical error on retained predictions does not exceed \varepsilon+O(1/\sqrt{n}) . 4. A multi-split evaluation protocol (random, epitope-held-out, distance-aware) paired with coverage–risk curves, ECE, and Brier score, which together surface failure modes hidden by standard single-metric evaluation. Experiments on a curated VDJdb–IEDB dataset confirm that calibration gains do not merely re-scale probabilities but enable operationally meaningful trade-offs: conformal abstention at 80% coverage reduces error rate by 41.7% relative to full-coverage prediction. II Related Work II-A TCR–pMHC Binding Prediction Early approaches relied on motif scoring and physicochemical features [ 3 , 17 ] . Subsequent neural models—including LSTM encoders, attention-based architectures, and graph neural networks—improved discrimination on random splits. TCRex [ 4 ] , TITAN [ 10 ] , and ERGO-II [ 19 ] represent increasingly expressive sequence models, and protein language model (pLM) features have improved sample efficiency [ 14 , 9 ] . However, these works evaluate primarily on random or epitope-in splits and do not address calibration or selective prediction. [ 8 ] and [ 18 ] demonstrate that performance degrades substantially under epitope-held-out evaluation, motivating shift-aware protocol design. II-B Calibration and Selective Prediction Calibration in deep learning was systematized by [ 5 ] , who showed that modern neural networks are systematically overconfident. Temperature scaling [ 5 ] remains the most effective single-parameter post-hoc correction. Selective prediction—abstaining when uncertainty exceeds a threshold—was analyzed theoretically by [ 1 ] under the coverage–risk framework. Conformal prediction provides distribution-free finite-sample guarantees [ 16 , 12 ] and has been adapted for selective classification [ 16 ] . Our work applies this framework specifically to biological sequence prediction under distribution shift, which introduces practical complications around calibration set construction and threshold validity. II-C Evaluation under Distribution Shift Shift-aware evaluation for biological sequence models has received increasing attention [ 6 , 13 ] . Distance-aware splits based on sequence similarity to the training set expose leakage that inflates random-split performance [ 15 , 11 ] . In the TCR–pMHC setting, epitope-held-out splits test cross-epitope generalization, the most practically relevant regime [ 2 , 7 ] . Our evaluation protocol combines both perspectives. III Method III-A Problem Formulation Let \tau\in\mathcal{A}^{*} denote a TCR CDR3 amino acid sequence (concatenation of CDR3 \alpha and CDR3 \beta ), and let \pi\in\mathcal{A}^{*} denote the peptide sequence of a pMHC complex. The binding prediction task is to learn a scoring function f_{\theta}:(\tau,\pi)\mapsto\hat{p}\in[0,1],\quad\hat{p}\approx P(y=1\mid\tau,\pi), (1) where y\in\{0,1\} indicates binding. The dataset is severely imbalanced ( \sim 4% positives) and evaluation occurs under three split regimes detailed in Section IV-A . III-B Dual-Encoder Architecture We encode TCR and peptide sequences independently using a shared pre-trained protein language model (ESM-2 650M parameters [ 9 ] ). For an input sequence s , ESM-2 produces per-token contextual embeddings; we apply mean pooling over non-padding positions to obtain a fixed-size representation \mathbf{h}_{s}\in\mathbb{R}^{1280} . The two representations are concatenated and passed through a two-layer MLP classifier with hidden dimension 512, layer normalization, GELU activations, and dropout ( p=0.2 ): z(\tau,\pi)=\text{MLP}([\mathbf{h}_{\tau};\mathbf{h}_{\pi}]),\quad\hat{p}=\sigma(z(\tau,\pi)), (2) where [\cdot;\cdot] denotes concatenation and \sigma is the sigmoid function. III-C Class-Weighted Training To handle label imbalance, we minimize a class-weighted binary cross-entropy loss: \mathcal{L}_{\text{train}}=-\frac{1}{n}\sum_{i=1}^{n}\bigl[w_{+}\,y_{i}\log\hat{p}_{i}+w_{-}\,(1-y_{i})\log(1-\hat{p}_{i})\bigr], (3) where w_{+}=n/(2n_{+}) and w_{-}=n/(2n_{-}) are inverse-frequency weights, n_{+} and n_{-} are positive and negative sample counts, and n=n_{+}+n_{-} . III-D Temperature Scaling Calibration Although the MLP classifier is trained end-to-end, its sigmoid outputs are typically miscalibrated under distribution shift. We apply post-hoc temperature scaling [ 5 ] : a single scalar T>0 is fitted on a split-matched calibration set \mathcal{D}_{\text{cal}} by minimizing the negative log-likelihood (NLL): T^{*}=\arg\min_{T>0}\sum_{(x,y)\in\mathcal{D}_{\text{cal}}}\mathcal{L}_{\text{CE}}\!\left(\sigma\!\left(\frac{z(x)}{T}\right),\,y\right). (4) The calibrated probability is \hat{p}_{T}(x)=\sigma(z(x)/T^{*}) . Calibration quality is measured by Expected Calibration Error (ECE) with M=15 equal-width bins: \text{ECE}=\sum_{m=1}^{M}\frac{|B_{m}|}{n}\bigl|\mathrm{acc}(B_{m})-\mathrm{conf}(B_{m})\bigr|, (5) where \mathrm{acc}(B_{m}) and \mathrm{conf}(B_{m}) are the mean accuracy and mean confidence of samples in bin B_{m} . III-E Conformal Selective Abstention To provide coverage guarantees at a user-specified target error rate \varepsilon , we employ split conformal prediction [ 16 , 12 ] . For a calibration example (x_{i},y_{i}) , the nonconformity score measures how poorly the calibrated model fits the true label: s_{i}=1-\hat{p}_{T}(x_{i})^{y_{i}}\cdot(1-\hat{p}_{T}(x_{i}))^{1-y_{i}}. (6) Given n_{\text{cal}} calibration scores, the conformal threshold at level \varepsilon is the \lceil(1-\varepsilon)(n_{\text{cal}}+1)\rceil -th smallest value: \hat{\tau}_{\varepsilon}=s_{(\lceil(1-\varepsilon)(n_{\text{cal}}+1)\rceil)}. (7) At test time, the model abstains when the nonconformity score exceeds \hat{\tau}_{\varepsilon} : \text{decision}(x)=\begin{cases}\arg\max\hat{p}_{T}(x)&\text{if }s(x)\leq\hat{\tau}_{\varepsilon},\\ \text{abstain}&\text{otherwise.}\end{cases} (8) By the exchangeability guarantee of split conformal prediction [ 16 ] , the empirical coverage satisfies P(s(x_{\text{test}})\leq\hat{\tau}_{\varepsilon})\geq 1-\varepsilon-1/(n_{\text{cal}}+1) when calibration and test data are exchangeable. The full procedure is summarized in

Read the complete text