LeanTTA: A Backpropagation-Free, Stateless Approach to On-Device Test-Time Adaptation
Cynthia Dong, Young D. Kwon, Hong Jia, George Rizos, Cecilia Mascolo · 2025
Read LeanTTA: A Backpropagation-Free, Stateless Approach to On-Device Test-Time Adaptation by Cynthia Dong, Young D. Kwon, Hong Jia, George Rizos, Cecilia Mascolo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15889.txt.
Introduction
Phones and wearables run learned models well until the world shifts and the data no longer match training. Cynthia Dong and co-authors present LeanTTA in their 2025 paper, a way for small devices to adapt at test time without the usual heavy cost. Their method updates only normalisation statistics, skips backpropagation entirely, and keeps no memory of past data.
LeanTTA also fuses quantised model parts and adapts only in part, which trims compute further. Across sensor data sets it beat leading rivals, cutting error by 15.7 percent against the best of them. A ResNet18 model needed just 11.2 megabytes at peak, and adaptation ran close to plain inference speed. The authors pitch it as a practical fix for models that must keep learning on weak hardware in changing places.
Catalog record
| CREATOR | Cynthia Dong, Young D. Kwon, Hong Jia, George Rizos, Cecilia Mascolo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-15889
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LeanTTA: A Backpropagation-Free, Stateless Approach to On-Device Test-Time Adaptation Cynthia Dong, Young D. Kwon, Hong Jia, George Rizos, Cecilia Mascolo Abstract For small, compressed models constrained by edge hardware and limited data storage or availability, test-time adaptation (TTA) can restore robustness against common corruptions that induce covariate shift. However, many TTA methods rely on backpropagation and large batch sizes, which we show fail under constrained data and hardware. In this work, we present LeanTTA , a combination of memory and compute-efficient tactics which allow us to deploy TTA on-device, even under quantized paradigms. LeanTTA consists of two components: (1) stateless, backpropagation-free statistics adjustment using an efficient calculation of the Mahalanobis distance and (2) partial adaptation through layerwise updates. LeanTTA reaches up to a 16% reduction in corruption error on CIFAR10 with minimal additional compute time and memory required over normal inference, demonstrating the potential of backpropagation-free methods for efficient on-device TTA. Code: https://github.com/cam-mobsys/LeanTTA . \workshoptitle AXIOM: Foundations of Efficient Deep Learning 1 Introduction Test-Time Adaptation (TTA) promises robust model deployment under distribution shifts, without access to data labels. For small models compressed to fit on edge hardware, TTA on-device can provide extra adaptability lost to quantization or compression—important, given that edge deployments encounter in-the-wild data which may be not only missing from the training set, but also highly limited in quantity, such as one recording of a rare bird under unusual conditions, or a single photo of a rare skin condition in a remote clinic [ 7 , 50 , 27 ] . However, mainstream approaches rely heavily on backpropagation and large batch sizes [ 52 ] , which become untenable within edge limitations, given memory constraints, the instability of quantized fine-tuning, and low data availability. We find in this work that small batch sizes can in fact collapse model accuracy to random guessing with most backpropagation-driven TTA methods, illustrating the incompatibility of most TTA methods and on-device deployment. To address this gap, we introduce LeanTTA , a combination of memory- and compute-efficient tactics which allow us to deploy TTA on-device, even under low-data and quantized paradigms. We first present stateless, backpropagation-free statistics adjustment via Mahalanobis distance, then demonstrate the accuracy and efficiency benefits of selective layerwise adaptation, which allow for fusion of unadapted layers for further computational gains. We evaluate on a common edge architecture (ResNet18 [ 16 ] , others in Appendix) and multiple datasets (ImageNet-C, CIFAR10-C [ 17 ] , and Warblr [ 50 ] ), benchmarking against SOTA methods on a resource-limited Raspberry Pi Zero 2W. Evaluated across corruption benchmarks, LeanTTA achieved up to a 16% reduction in corruption error ( \frac{Adaptation-Corruption}{Baseline-Corruption} , calculated on CIFAR-10 with a baseline of 93%) across both gradual and abrupt shifts while operating within an 11.2 MB peak memory footprint on ResNet18, delivering an order-of-magnitude reduction in memory consumption over similarly-performing TTA methods. 2 Background and Related Works TTA aims to reduce the effect of test-time covariate shift through fine-tuning or direct manipulation of the source statistics frozen in batch normalization layers at test time [ 27 ] . Normalization layers ensure that intermediate activations arrive at subsequent layers with a standardized mean and variance [ 21 , 46 ] ; when deployment-time data arrives with shifted statistics, frozen training statistics cannot standardize activations into the operating region expected by other layers [ 33 ] . The early work Tent [ 52 ] and many variants [ 53 , 44 , 43 , 31 ] rely on entropy-supervised backpropagation, where batch normalization statistics are continuously updated using gradient descent, supervised by entropy (and thereby confidence) as a proxy for accuracy drops due to covariate shift. Methods like [ 43 , 31 , 44 ] iterate on entropy-backpropagation methods through selective gradient updates, while methods like [ 53 , 58 , 15 ] aim to decrease model collapse and error accumulation due to continual adaptation. For example, MECTA [ 19 ] aims to reduce memory through pruning and selective activation caching, and TinyTTA [ 25 ] through early-exit ensembles on microcontrollers. These works require backwards passes and gradient calculations—untenable with limited compute. Forward-only adaptation as proposed in [ 33 , 51 , 42 , 57 ] improves memory- and compute efficiency. TEMA [ 51 ] estimates statistics over previous test batches, FOA [ 42 ] relies on a particular architecture, and PEA [ 57 ] lifts this restriction yet relies on an exponential moving average of test-time statistics. LeanTTA builds on the efficiency of forward-only methods and adds several features. To mitigate degradation from continual adaptation, we demonstrate that adaptation relative to original source statistics per-instance is sufficient to improve accuracy. LeanTTA also leverages for more compute efficiency the finding that shallow-layer adaptation matches or exceeds full-model adaptation, and proposes fusion of non-adaptive layers for further memory and compute savings (see extended discussion in Appendix A.1 ). 3 The Method Problem Description: A model f_{\theta}(\textbf{x}) is trained over a source dataset with inputs and labels x_{s},y_{s} , sampled from the joint probability distribution P_{source}(x_{s},y_{s}) . At test-time, the model f_{\theta}(\textbf{x}) with parameters \theta encounters inputs x_{t} with unknown labels y_{t} , where x_{t},y_{t} are distributed over P_{target}(x_{t},y_{t}) , and P_{target}(x_{t},y_{t}) may have diverged from P_{source}(x_{s},y_{s}) . The model f_{\theta}(\textbf{x}) must adapt to any differences between P_{source} and P_{target} with access only to x_{t} . Given a pretrained model, LeanTTA updates and adapts normalization layers to target statistics while maintaining its performance on the source domain. We discuss the four key steps below. Extract Statistical Information: We extract statistical information from the input to each normalization layer x_{t}^{l} . We then calculate the mean ( \mu_{t}\in\mathbb{R}^{c} ) and variance ( \sigma_{t}^{2}\in\mathbb{R}^{c} ): \mu_{t}\leftarrow\frac{1}{H\times W}\sum x_{t}^{l} , \sigma_{t}^{2}\leftarrow Var(x_{t}^{l}) . H and W represent the height and width of the input data per-channel, respectively, where c is the number of channels. The calculated target statistics are combined with the saved source statistics at that layer (\mu_{s},\sigma_{s}^{2}) using a weighted average, parameterized by \tau : \mu_{b}\leftarrow\tau\mu_{s}+(1-\tau)\mu_{t} , \sigma_{b}^{2}\leftarrow\tau\sigma_{s}^{2}+(1-\tau)\sigma_{t}^{2} . Target statistics, though informative, are highly variable between single data samples and cannot directly replace source statistics. Instead, we use an estimated moving average via \tau , which determines the weight assigned to the source ( \tau ) and the target statistics ( 1-\tau ). Calculate Sample Divergence: To effectively adapt to single data points, we calculate a distance measure, d , incorporating Mahalanobis distance [ 10 ] to measure the divergence between source and our now-stabilized target distributions: d\leftarrow 1-e^{-(\mu_{b}-\mu_{s})^{T}(\Sigma_{s}^{2})^{-1}(\mu_{b}-\mu_{s})} . Composing the \Sigma_{s}^{2} as a diagonal variance matrix \in\mathbb{R}^{c\times c} , inversion can be replaced by faster simple division. We apply Mahalanobis distance as it accounts for the variance structure of both source and target distributions, and has been applied in SOTA out-of-distribution detection tasks. Balance Source and Target: The Mahalanobis distance indicates distribution shift severity, informing whether to give more weight to the source or stabilized incoming statistics. For severe shifts, more emphasis is placed on source statistics; less-severe shifts place more emphasis on stabilized incoming statistics: \mu_{\text{new}}\leftarrow d\lambda\mu_{s}+(1-d\lambda)\mu_{b} , \sigma_{\text{new}}^{2}\leftarrow d\lambda\sigma_{s}^{2}+(1-d\lambda)\sigma_{b}^{2} . \lambda ensures that even when the Mahalanobis distance d is 1, some of the target statistic is still incorporated to improve accuracy and ensure a smooth transition between source