BP-TTA: Balanced and Prototype-Guided Test-Time Adaptation in Dynamic Scenarios
Shaoyang Huang, Yashi Zhu, Yichen Yu, Lei Zhang, Zhang Yi, Tao He · 2026
Read BP-TTA: Balanced and Prototype-Guided Test-Time Adaptation in Dynamic Scenarios by Shaoyang Huang, Yashi Zhu, Yichen Yu, Lei Zhang, Zhang Yi, Tao He (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31420.txt.
Introduction
Huang and five colleagues present BP-TTA, a test-time adaptation method for streams where classes arrive unevenly. Test-time adaptation updates a model on unlabeled test data as domains shift. In live streams, class imbalance and domain shift arrive together and feed each other, yet most methods handle drift alone.
BP-TTA builds balanced adaptation batches by mixing current samples with high-confidence past instances, which curbs bias toward common classes. It also keeps evolving class prototypes during inference and uses prototype similarity to discipline the updates, making pseudo-labels safer. Across dynamic streaming tests the method beats leading rivals steadily. The pairing of balanced batches with prototype guidance keeps online learning stable when both the world and the class mix keep moving.
Catalog record
| CREATOR | Shaoyang Huang, Yashi Zhu, Yichen Yu, Lei Zhang, Zhang Yi, Tao He |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.31420
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BP-TTA: Balanced and Prototype-Guided Test-Time Adaptation in Dynamic Scenarios Shaoyang Huang, Yashi Zhu, Yichen Yu, Lei Zhang, Zhang Yi, Tao He Abstract Test-Time Adaptation (TTA) enables models trained on a source domain to adapt online to unlabeled test data under distribution shifts. While recent TTA methods have moved beyond static settings and begun to consider continual domain shifts, they primarily address distribution drift and fail to account for class imbalance in dynamic scenarios. In real-world test-time streams, class imbalance and continual domain shifts often occur at the same time and interact with each other. In this paper, we propose a novel Balanced and Prototype-Guided Test-Time Adaptation (BP-TTA) method, which combines batch-balanced sampling with prototype-guided adaptation to handle the class imbalance and continual domain shift problems. BP-TTA constructs balanced adaptation batches by integrating current samples with high-confidence historical instances, effectively mitigating bias toward dominant classes and stabilizing online updates. Meanwhile, BP-TTA maintains evolving class prototypes during inference and leverages prototype similarity as a constraint for model adaptation, thereby improving the reliability of pseudo-labels and enhancing the stability of online updates under persistent domain shifts. Extensive experiments demonstrate that BP-TTA consistently outperforms state-of-the-art TTA methods in dynamic test-time streaming settings. I Introduction Deep neural networks have achieved remarkable progress in computer vision and related fields in recent years [ 10 , 19 ] . However, their success typically relies on the assumption that training and test data follow the same distribution, which rarely holds in real-world applications. In practice, factors such as illumination changes, weather variations, sensor noise, and differences in imaging devices often lead to distribution shifts. When faced with such shifts at inference time, model performance can degrade substantially, sometimes resulting in catastrophic failures [ 9 ] . Since it is infeasible to train models on all possible data distributions, this reliance on the training distribution limits their adaptability. To mitigate distribution shifts, prior research has extensively explored Unsupervised Domain Adaptation (UDA), which addresses domain gaps by jointly training on labeled source data and unlabeled target data [ 6 , 21 , 20 , 24 ] . However, it requires access to both source and target data during training. In practice, target data is often unavailable beforehand, distribution shifts are unpredictable, and access to source data may also be restricted due to privacy or computational constraints. In response to this challenge, Test-Time Adaptation (TTA) has recently attracted growing attention in scenarios involving domain shifts caused by image corruptions [ 4 , 18 ] . In this paradigm, a pre-trained source model, without access to its training data, is adapted to unlabeled test data through online training. Early TTA studies [ 39 , 30 , 2 ] primarily focused on static adaptation settings. These methods assumed that all test samples were independently drawn from a single fixed target domain, implying that each test batch was independently and identically distributed (i.i.d.). Under this assumption, the model can gradually adapt to a stationary target distribution by minimizing entropy or updating normalization statistics. However, such an assumption is often unrealistic in practice, as real-world test data rarely follow a fixed distribution and instead exhibit complex and evolving patterns over time. To address this limitation, later works [ 40 , 26 , 42 ] proposed Continual Test-Time Adaptation (CTTA), which considers a more realistic dynamic adaptation scenario where the test distribution continuously shifts over time under changing environments. Furthermore, [ 47 ] introduced a more practical setting that not only follows the continual adaptation paradigm but also accounts for the fact that test data usually arrive as streaming sequences with strong temporal correlations. This often leads to severe class imbalance within short time windows. Termed Practical Test-Time Adaptation (PTTA), this setting presents two critical challenges: class imbalance and continuous domain shifts. First, class imbalance occurs when some classes appear frequently within a short period while others are underrepresented or temporarily absent. This situation often comes from temporal correlation in real-world streaming data. For example, in autonomous driving, vehicles on highways predominantly observe cars, whereas in residential areas they mostly encounter pedestrians or non-motorized vehicles. Such temporally localized distributions lead to biased model updates because the model repeatedly learns from dominant classes [ 32 ] . As a result, gradient updates are dominated by majority classes, causing the model to overfit to short-term patterns while neglecting minority classes, ultimately degrading overall generalization performance. Second, continuous domain shift refers to gradual changes in the test distribution as environmental conditions evolve over time. For instance, in autonomous driving, lighting changes when vehicles move from sunny roads to rainy streets or tunnels, causing shifts in input features over time [ 35 ] . Under continual domain shift settings, TTA methods that rely on pseudo-labels or entropy minimization tend to be unstable and highly sensitive to noisy signals. As the distribution drifts, pseudo-labels may degrade, producing noisy supervision that misguides gradient updates and reduces model reliability. To jointly address class imbalance and continuous domain shift, we propose a Balanced and Prototype-Guided Test-Time Adaptation (BP-TTA) method, which combines batch-balanced sampling with prototype-guided adaptation. The following are our main contributions: • We propose Batch-Balanced Sampling (BBS) to address class imbalance, which constructs a class-balanced memory bank from realistic test streams and dynamically reconstructs the current batch according to the class distribution of incoming samples, effectively mitigating class imbalance. The details are illustrated in Fig. 1a . This design ensures that each model update is guided by high-quality, balanced pseudo-labels, thereby improving adaptation robustness in dynamic streams. • We propose Category Prototype-Guided Adaptation (CPGA) to handle continuous domain shifts, aiming to improve pseudo-label quality and stabilize TTA. As shown in Fig. 1b , CPGA dynamically maintains class prototypes during testing and incorporates feature-prototype similarity into model updates, enabling active adaptation in the feature space. • We evaluate our method on three datasets under data streams with coexisting class imbalance and continually changing distributions, demonstrating superior performance over recent state-of-the-art methods. (a) (b) Fig. 1: Visualization of our two key components. (a) BBS selects high-quality samples from majority classes and supplements minority-class samples from the memory bank to construct a balanced batch for adaptation. The memory bank is selectively updated with new samples from the current batch. (b) CPGA first updates class prototypes using high-confidence samples’s features (solid triangles), then computes the prototype loss to guide model adaptation, promoting intra-class clustering and inter-class separation. II Related Work II-A Test-Time Adaptation Neural networks have become the leading topic of artificial intelligence, and recurrent models [ 25 , 14 , 13 ] and ODE-based methods [ 45 , 46 , 12 ] have been largely developed for complicate machine learning tasks. However, these models performed not well in dynamic scenarios with continual domain shifts. Unsupervised Domain Adaptation (UDA) aims to train a model using labeled source domain data and generalize it to an unlabeled target domain, addressing the challenge of distribution shift without requiring target domain annotations [ 6 , 21 , 20 , 24 ] . However, it may be infeasible when training data are unavailable due to privacy or storage concerns. To overcome this limitation, [ 39 ] proposed Test-Time Adaptation (TTA), which focuses on real-time model adjustment during the testing phase using unlabeled