Foundation Models Defining A New Era In Sensor-based Human Activity Recognition: A Survey And Outlook DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper
Sizhen Bian, Mengxi Liu, Lala Shakti Swarup Ray, Bo Zhou, Bin Guo, Zhiwen Yu, Thomas Plötz, Paul Lukowicz, Siyu Yuan, Vitor Fortes Rey · 2026
Read Foundation Models Defining A New Era In Sensor-based Human Activity Recognition: A Survey And Outlook DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper by Sizhen Bian, Mengxi Liu, Lala Shakti Swarup Ray, Bo Zhou, Bin Guo, Zhiwen Yu, Thomas Plötz, Paul Lukowicz, Siyu Yuan, Vitor Fortes Rey (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-02711.txt.
Introduction
Bian and nine colleagues survey foundation models for sensor-based human activity recognition, the task of reading what a person is doing from wearable sensors. Current models struggle with scarce labels, varied sensors, and weak transfer across users and devices. The survey sorts work by a lifecycle taxonomy, tracks nine design axes, and names three development paths, from models trained from scratch on sensor data to adapted general time-series models.
Catalog record
| CREATOR | Sizhen Bian, Mengxi Liu, Lala Shakti Swarup Ray, Bo Zhou, Bin Guo, Zhiwen Yu, Thomas Plötz, Paul Lukowicz, Siyu Yuan, Vitor Fortes Rey |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 104130 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.02711
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Foundation Models Defining A New Era In Sensor-based Human Activity Recognition: A Survey And Outlook DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper Sizhen Bian, Mengxi Liu, Lala Shakti Swarup Ray, Bo Zhou, Bin Guo, Zhiwen Yu, Thomas Plötz, Paul Lukowicz, Siyu Yuan, Vitor Fortes Rey Abstract . Sensor-based Human Activity Recognition (HAR) underpins many ubiquitous and wearable computing applications, yet current models remain limited by scarce labels, sensor heterogeneity, and weak generalization across users, devices, and contexts. Foundation models, which are generally pretrained at scale using self-supervised and multimodal learning, offer a unifying paradigm to address these challenges by learning reusable, adaptable representations for activity understanding. This survey synthesizes emerging foundation models for sensor-based HAR. We first clarify foundational concepts, definitions, and evaluation criteria, then organize existing work using a lifecycle-oriented taxonomy spanning input design, pretraining, adaptation, and utilization. Rather than enumerating individual models, we analyze recurring design patterns and trade-offs across nine technical axes, including modality scope, tokenization, architectures, learning paradigms, adaptation mechanisms, and deployment settings. From this synthesis, we identify three dominant development trajectories: (i) HAR-specific foundation models trained from scratch on large sensor corpora, (ii) adaptation of general time-series or multimodal foundation models to sensor-based HAR, and (iii) integration of large language models for reasoning, annotation, and human–AI interaction. We conclude by highlighting open challenges in data curation, multimodal alignment, personalization, privacy, and responsible deployment, and outline directions toward general-purpose, interpretable, and human-centered foundation models for activity understanding. A complete, continuously updated index of papers and models is available in our companion repository 1 1 1 https://github.com/zhaxidele/Foundation-Models-Defining-A-New-Era-In-Human-Activity-Recognition . 1. Introduction Sensor-based Human Activity Recognition (HAR) has long been a cornerstone of ubiquitous and wearable computing, enabling applications in health monitoring ( 3 ; 75 ) , human–computer interaction ( 26 ; 15 ) , and smart environments ( 2 ; 66 ) . Despite the rapid evolution of deep learning, most HAR models remain limited by their reliance on task-specific architectures, small annotated datasets, and poor generalization across users, devices, or contexts ( 8 ; 7 ; 61 ; 69 ) . Recently, the emergence of foundation models has suggested a viable route to mitigate HAR’s core constraints, fundamentally transforming how activity understanding can be approached from wearable, ambient, and physiological sensor data ( 79 ; 118 ; 171 ) . As illustrated in Figure 1 , the development of sensor-based HAR has followed a clear trajectory, from classical machine learning based on handcrafted features, through deep, transferable, and self-supervised learning, to today’s emerging foundation models. This evolution reflects a continuous pursuit of scalability, generalization, and interpretability in understanding human activity from sensor data ( 39 ; 65 ) . Figure 1 . Historical development of sensor-based Human Activity Recognition (HAR) models. From classical machine learning with hand-crafted features and shallow classifiers ( 12 ) to the rise of deep learning with CNNs and RNNs ( 112 ) , the field progressed toward a phase focused on transfer and domain generalization (robustness across users, devices, and datasets ( 129 ; 122 ; 143 ) ). More recently, self-supervised learning (SSL) approaches have enabled pretraining on unlabeled sensor data using contrastive or masked objectives ( 142 ) . Today, the field is moving toward foundation models, exemplified by large-scale sensor–language alignment ( 125 ) , emphasizing scalability, generalization, and interpretability. The rising interest in sensor-based HAR foundation models stems from several converging developments: (1) the proliferation of constantly emerging wearable and ambient datasets capturing multimodal physiological, kinematic, and contextual signals ( 164 ; 128 ; 57 ) ; (2) advances in self-supervised learning techniques such as masked autoencoding and contrastive alignment ( 54 ; 63 ; 30 ) ; (3) the adaptation of transformer architectures to sequential and multimodal sensor streams ( 150 ; 62 ; 132 ) ; and (4) the emergence of cross-modal and sensor–language models that bridge raw sensor signals and semantic concepts ( 170 ; 92 ; 79 ; 40 ; 89 ; 105 ) . These trends collectively extend the scaling behaviors and representational richness once confined to vision and language domains into the realm of sensor-based HAR. Motivated by these developments, this survey provides the first comprehensive synthesis of foundation models for sensor-based HAR. While prior surveys, such as ( 55 ; 16 ) , have reviewed the general evolution and trends of sensor-based HAR, they have not examined foundation models as a significantly distinct paradigm nor analyzed their emerging design principles and domain-specific requirements. The earliest work closely aligned with the concept of foundation models in sensor-based HAR is LIMU-BERT ( 153 ) , introduced in 2021. This pioneering model leveraged self-supervised learning to extract generalized, task-agnostic representations from unlabeled IMU data, inspired by the pretraining principles of the BERT language model to effectively capture temporal relations and statistical distributions within inertial signals. In 2022, the pretrain-then-finetune paradigm grounded in self-supervised learning gained rapid momentum as researchers sought to mitigate the data scarcity challenge in sensor-based HAR ( 54 ) . These early efforts paved the way for the current wave of foundation models by emphasizing label-efficient learning, multimodal alignment, and transferable representations. As a result, the community has observed a steep upward trend in foundation models for sensor-based HAR, with the majority of studies emerging since 2024. This work analyzes 132 representative papers and organizes the field using a unified, lifecycle-oriented taxonomy spanning input design, pretraining, adaptation, and utilization, structured across nine technical and operational axes. Building on this taxonomy, this work identifies three dominant development directions: (1) developing HAR-specific foundation models from scratch, (2) adapting general time-series and multimodal foundation models to sensor data, and (3) leveraging large language models for reasoning, annotation, and human–AI interaction. Open challenges are further discussed, including large-scale data curation, multimodal alignment, on-device personalization, and responsible deployment, and future research directions are outlined toward robust, generalizable, and human-centered foundation models for activity understanding. It should be noted that this work focuses specifically on sensor-based HAR and deliberately excludes vision- or video-based HAR from the scope of this survey. Here, sensor-based HAR refers to recognition approaches based on non-visual sensing modalities, including inertial sensors (e.g., accelerometers and gyroscopes), physiological sensors (e.g., ECG, PPG), RF signals, acoustic sensing, and other ambient or wearable sensing technologies. Vision-based HAR, including approaches relying on RGB/depth cameras, wearable cameras, and vision–language models (VLMs), represents a distinct and well-established research direction with its own datasets, benchmarks, and modeling paradigms. While such methods are highly relevant to the broader field of ubiquitous computing, they differ fundamentally from sensor-based HAR. In particular, challenges such as data scarcity and annotation complexity are generally less severe in vision-based approaches due to the availability of large-scale datasets and mature benchmarks, whereas sensor-based HAR often operates with limited, heterogeneous, and less standardized data. As a result, the two paradigms