Parameter-Efficient Fine-Tuning for HAR: Integrating LoRA and QLoRA into Transformer Models

Irina SEREGINA, Philippe LALANDA, German VEGA · 2025

Read Parameter-Efficient Fine-Tuning for HAR: Integrating LoRA and QLoRA into Transformer Models by Irina SEREGINA, Philippe LALANDA, German VEGA (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-17983.txt.

Introduction

Irina Seregina, Philippe Lalanda, and German Vega test cheap ways to adapt big activity spotting models to new data. Human activity recognition powers many health and home tools, yet full retraining of a large model is too heavy for small devices. They try two light add on methods, LoRA and its compressed cousin QLoRA, on a transformer backbone.

Across five public data sets, each tested as the held out newcomer in turn, both methods matched full fine tuning scores. They used far fewer trained parts, less memory, and less time. The size of the add on gave a clean dial between accuracy and cost. QLoRA squeezed the frozen weights further with little loss in quality.

Catalog record

CREATORIrina SEREGINA, Philippe LALANDA, German VEGA
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-17983

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Parameter-Efficient Fine-Tuning for HAR: Integrating LoRA and QLoRA into Transformer Models Irina SEREGINA, Philippe LALANDA, German VEGA Abstract Human Activity Recognition (HAR) is a foundational task in ubiquitous computing with applications in health monitoring, smart environments, and human–computer interaction. While recent advances in self-supervised learning and transformer-based architectures have significantly improved HAR performance, adapting large pretrained models to new domains remains a practical challenge due to limited computational resources on target devices. This papers investigates parameter-efficient fine-tuning techniques, specifically Low-Rank Adaptation (LoRA) and Quantized LoRA, as scalable alternatives to full model fine-tuning for HAR. We propose an adaptation framework built upon a Masked Autoencoder backbone and evaluate its performance under a Leave-One-Dataset-Out validation protocol across five open HAR datasets. Our experiments demonstrate that both LoRA and QLoRA can match the recognition performance of full fine-tuning while significantly reducing the number of trainable parameters, memory usage, and training time. Further analyses reveal that LoRA maintains robust performance even under limited supervision and that the adapter rank provides a controllable trade-off between accuracy and efficiency. QLoRA extends these benefits by reducing the memory footprint of frozen weights through quantization, with minimal impact on classification quality. I Introduction Deep learning has become a cornerstone of modern artificial intelligence, particularly in pervasive computing applications [ 2 ] . Unlike traditional machine learning approaches that rely on handcrafted features, deep models automatically extract representations from raw sensor time series, making them especially effective for high-dimensional and heterogeneous data. A major breakthrough was the introduction of Transformers—self-attention architectures that capture long-range dependencies without recurrence [ 34 ] . Originally designed for natural language processing, they have since been adapted to sensor-based time-series tasks [ 8 ] . Their scalability and representational power make them attractive for pervasive computing; however, they typically require large training datasets to converge and involve millions of parameters, which limits their deployment on resource-constrained devices. Moreover, in pervasive computing, labeled data are often scarce due to costly annotation and high context variability, further exacerbating the data-hungry nature of Transformers. To mitigate label scarcity, practitioners increasingly adopt a two-stage pipeline: pre-training on large, heterogeneous corpora followed by task-specific fine-tuning. Pre-training yields transferable, general-purpose features, while fine-tuning adapts them to downstream tasks with limited supervision. This approach has proved effective in many pervasive applications—for example, self-supervised pre-training on large wearable collections before Human Activity Recognition (HAR) adaptation [ 9 ] , or pre-training on multi-sensor sleep datasets prior to specialization on smaller clinical cohorts [ 16 ] . Beyond representation transfer, pre-training effectively enlarges the usable training signal by leveraging related—though not identical—domains and contexts. This label efficiency is especially valuable for Transformers, whose strong capacity entails substantial data requirements. The purpose of fine-tuning is to adapt a (pretrained) model to a target domain by updating its parameters. Full fine-tuning, where all parameters are adjusted, often yields strong performance but comes at high computational and memory costs, which limits its applicability to resource-constrained devices. To address this, parameter-efficient fine-tuning (PEFT) methods have been proposed. Techniques such as Low-Rank Adaptation (LoRA) or adapters update only a small fraction of additional parameters, while keeping most pretrained weights frozen. This approach reduces training and storage costs substantially while preserving accuracy and reducing the amount of labeled data required for adaptation, making it attractive for deployment in real-world settings. In this paper, we study human activity recognition and incorporate Low-Rank Adaptation (LoRA) into a Transformer backbone pretrained via Masked Autoencoding (MAE). We show that this integration is both feasible and effective, offering a practical path to scalable, low-cost—and data-efficient—personalization in HAR and, more broadly, pervasive computing. The choice of HAR as our application domain is motivated by the growing need for systems that robustly adapt to non-stationary, real-world conditions. As deployments scale across users, devices, and contexts, HAR models must deliver accurate, personalized predictions on resource-constrained mobile and wearable platforms—often with limited labeled data and under strict privacy constraints. Conventional supervised models, even when pretrained, often fail to generalize without costly full-model adaptation. We further extend the study to QLoRA, which introduces quantization to reduce memory footprint and improve inference speed, enabling practical deployment on edge and IoT devices. More specifically, we investigate whether integrating Transformers and (Q)LoRA fine-tuning enhances the adaptability of HAR models under domain shifts and low-resource conditions. We benchmark the proposed approach on multiple publicly available HAR datasets and evaluate its performance using a Leave-One-Dataset-Out validation protocol across five heterogeneous HAR benchmarks. The key contributions of this work are then as follows: • We introduce the integration of LoRA into transformer-based HAR models pretrained with MAE, enabling parameter-efficient fine-tuning for domain adaptation. • We extend this approach with QLoRA, reducing the memory footprint of frozen weights through quantization and enabling faster, more resource-efficient deployment. • We conduct extensive experiments under a Leave-One-Dataset-Out validation protocol across five heterogeneous HAR datasets, demonstrating that LoRA maintains robust performance while providing a tunable trade-off between accuracy and efficiency via adapter rank. QLoRA further enhances deployability by lowering resource requirements with minimal impact on accuracy. The paper is structured as follows: Section 2 reviews related work on HAR, and fine-tuning strategies with a focus on parameter-efficient adaptation, including LoRA and QLORA. Section 3 describes the proposed methodology, including the model architecture, the integration of LoRA and QLoRA, and dataset preprocessing. Section 4 describes the implementation and section V presents experimental results and comparative analyses between full fine-tuning, LoRA, and QLoRA. Finally, Section 5 concludes the paper with a summary of findings and directions for future work. II Background and related work II-A Human Activity Recognition Human Activity Recognition (HAR) is the automatic identification of physical activities from data collected by wearable or ambient sensors such as accelerometers and gyroscopes cite [ 15 , 32 ] . It supports applications in healthcare, personalized fitness, rehabilitation, workplace safety, and smart homes. Over the years, HAR has progressed from shallow classifiers with handcrafted features to deep learning models such as CNNs, RNNs, and transformers, which can learn spatiotemporal representations directly from raw multivariate signals [ 17 ] . Yet HAR suffers from pronounced domain shift: models trained on one dataset often degrade on unseen users, devices, placements, sampling rates, or behaviors [ 25 ] . In HAR, domain shifts commonly arise due to: • Covariate shift: the input distribution P(x) changes while P(y\!\mid\!x) remains the same (e.g., older adults walking more slowly than younger adults); • Label shift: the label distribution P(y) changes while P(x\!\mid\!y) remains the same (e.g., datasets dominated by low- vs. high-intensity activities); • Concept shift: the relationship P(y\!\mid\!x) changes (e.g., identical motion patterns labeled differently across contexts). Hence, even with strong pretrained representations, fine-tuning is fundamental to adapt models to the target domain (user, device, placement) and to recover robustness under heterogeneity and noise. II-B Fine Tuning: basics Full fine-tuning is the process of updating all parameters of a pretrained neural network when adapting it to a new task or domain (as usually defined in SE [ 18 , 10 ] ). It remains the standard and most flexible method for adapting large models, especially when there is a significant mismatch between the source and target data distributions. A typical full-adaptation pipeline is: (1) initialize a pretrained backbone

Read the complete text