Beyond Physical Labels: Redefining Domains for Robust WiFi-based Gesture Recognition DOI: XXXXXXX.XXXXXXX Journal: IMWUT Volume: 0 0 0 8 CCS: Human-centered computing Ubiquitous and mobile computing CCS: Human-centered computing Interaction techniques
Xiang Zhang, Huan Yan, Jinyang Huang, Bin Liu, Yuanhao Feng, Jianchun Liu, Meng Li, Fusang Zhang, Zhi Liu · 2026
Read Beyond Physical Labels: Redefining Domains for Robust WiFi-based Gesture Recognition DOI: XXXXXXX.XXXXXXX Journal: IMWUT Volume: 0 0 0 8 CCS: Human-centered computing Ubiquitous and mobile computing CCS: Human-centered computing Interaction techniques by Xiang Zhang, Huan Yan, Jinyang Huang, Bin Liu, Yuanhao Feng, Jianchun Liu, Meng Li, Fusang Zhang, Zhi Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03825.txt.
Introduction
Xiang Zhang and eight co-authors present GesFi, a gesture recognition system that works from ordinary WiFi signals and survives moving to new environments. WiFi-based sensing usually breaks when the room or user changes, a problem blamed on domain shift. Instead of defining domains by physical labels like room or person, GesFi mines latent domains directly from the data itself.
Raw signals are cleaned with ratio denoising, transformed, and visualized into standard inputs. Class-wise adversarial learning suppresses gesture identity where it should not matter, while unsupervised clustering uncovers the hidden factors causing distribution shifts. Those latent domains are then aligned adversarially for robust generalization. Tests across public datasets and real rooms, with one or several WiFi pairs, beat leading baselines, pointing toward camera-free gesture control that travels between homes.
Catalog record
| CREATOR | Xiang Zhang, Huan Yan, Jinyang Huang, Bin Liu, Yuanhao Feng, Jianchun Liu, Meng Li, Fusang Zhang, Zhi Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03825
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Physical Labels: Redefining Domains for Robust WiFi-based Gesture Recognition DOI: XXXXXXX.XXXXXXX Journal: IMWUT Volume: 0 0 0 8 CCS: Human-centered computing Ubiquitous and mobile computing CCS: Human-centered computing Interaction techniques Xiang Zhang, Huan Yan, Jinyang Huang, Bin Liu, Yuanhao Feng, Jianchun Liu, Meng Li, Fusang Zhang, Zhi Liu Abstract . WiFi-based gesture recognition holds great promise due to its non-intrusive and ubiquitous nature. However, robust generalization across diverse conditions remains a core challenge due to the inherent sensitivity and ambiguity of Channel State Information (CSI) signals, making them unreliable for real-world human-computer interaction (HCI) applications. While domain adversarial learning-based systems have proven effective in other sensing modalities, its performance in WiFi-based gesture recognition remains limited. In this paper, we revisit the foundational assumptions in current adversarial learning-based WiFi gesture recognition systems and identify a key mismatch: conventional methods rely on subjective physical domain labels (e.g., location, orientation), which often fail to reflect true distributional patterns in the data. This misalignment introduces two critical issues, namely classification conflict and manifold distortion, which fundamentally limit generalization performance. To address these challenges, we propose GesFi , a novel WiFi-based gesture recognition system that introduces WiFi latent domain mining to redefine domains directly from the data itself. GesFi first processes raw sensing data collected from WiFi receivers using CSI-ratio denoising, Short-Time Fast Fourier Transform, and visualization techniques to generate standardized input representations. It then employs class-wise adversarial learning to suppress gesture semantic and leverages unsupervised clustering to automatically uncover latent domain factors responsible for distributional shifts. These latent domains are then aligned through adversarial learning to support robust cross-domain generalization. Finally, the system is applied to the target environment for robust gesture inference. We deployed GesFi under both single-pair and multi-pair settings using commodity WiFi transceivers, and evaluated it across multiple public datasets and real-world environments. Compared to state-of-the-art baselines, GesFi achieves up to 78% and 50% performance improvements over existing adversarial methods, and consistently outperforms prior generalization approaches across most cross-domain tasks. (Code available at https://github.com/CamLoPA/GesFiCode ). 1. Introduction Gesture recognition systems ( 15 ) play a pivotal role in the field of human-computer interaction. In recent years, WiFi-based gesture recognition systems have garnered significant attention due to its universal deployment capabilities and non-intrusive nature ( 9 ; 5 ) , and the integration of machine learning has further enhanced the performance of these systems. However, WiFi sensing frequently faces major generalization challenges ( 26 ; 32 ) . At the core of WiFi-based gesture recognition is the sensing of limb-induced changes in signal reflection paths, captured through Channel State Information (CSI) ( 22 ; 45 ; 28 ) . This mechanism is intrinsically low-dimensional, as it compresses the three-dimensional spatial dynamics of hand movements into a one-dimensional representation of reflection path length. Consequently, WiFi sensing exhibits two critical characteristics: high sensitivity to various factors (e.g., location, orientation) and significant semantic ambiguity. A single gesture may produce dramatically different CSI patterns under varying conditions, while distinct gestures may yield similar CSI traces. These properties make domain shifts in WiFi sensing particularly complex. Figure 1. Prior approaches ( 17 ) typically assume that distributional shifts can be captured by physical labels, such as environments. However, we argue that physical domains are insufficient to represent cross-domain distributional variations in WiFi sensing. For example, within a single environment, a push gesture performed in opposite directions can result in entirely different CSI patterns, forming multiple disconnected distributional clusters. Conversely, push and pull gestures executed in different environments may exhibit highly similar CSI representations, leading to distributional confusion. In some cases, the variation in CSI within a single physical domain may exceed that between different domains. A straightforward approach to enhancing generalization would involve collecting large and diverse WiFi sensing datasets that capture a wide range of potential environmental and user conditions. However, collecting and annotating wireless sensing data is highly cumbersome compared to visual or auditory data, severely limiting the scale and diversity achievable in practice. Consequently, recent research has increasingly focused on developing generalized gesture recognition systems using limited training data. ( 31 ) . Existing strategies fall into three broad categories: domain-invariant feature extraction, domain adaptation, and domain generalization. Domain-invariant feature extraction leverages expert knowledge to manually design features less sensitive to domain shifts, such as the Body-coordinate Velocity Profile (BVP) ( 55 ) and the Motion Navigation Primitive (MNP) ( 10 ) . However, such handcrafted features often lead to non-negligible information loss compared to the original signal. Domain adaptation ( 56 ; 48 ) enables gesture recognition systems to adjust to target domains via distribution alignment, assuming access to target domain data. Yet, even when only a few samples per class are required ( 32 ; 7 ) , the overall data collection burden remains substantial in complex WiFi environments. In contrast, domain generalization aims to build gesture recognition systems that can generalize effectively to unseen domains using limited training data, by leveraging the powerful inductive capabilities of machine learning. Existing approaches have primarily pursued this goal by designing more expressive architectures capable of extracting richer signal representations ( 26 ; 12 ) , or by augmenting the source domain distribution through synthetic or transformed data ( 14 ; 6 ) . Some methods have also adopted domain adversarial learning as a means to achieve domain generalization in WiFi-based gesture recognition ( 17 ; 24 ; 13 ) . These approaches typically rely on domain labels to distinguish source domains and employ a minimax game between a feature extractor and a domain discriminator to learn domain-invariant representations. Due to its intuitive mechanism for eliminating domain-specific factors, along with its theoretical guarantees and framework flexibility, domain adversarial learning has become a widely used strategy across various sensing modalities. However, multiple recent studies ( 26 ; 52 ; 25 ; 43 ) have reported suboptimal performance when applying adversarial-based methods to WiFi gesture recognition ( 17 ; 24 ; 13 ) . This naturally leads to the following question: Q1: Why do adversarial-based WiFi gesture recognition systems exhibit limited effectiveness? We posit that the limited effectiveness of existing approaches stems from a fundamental misconception regarding the notion of "domain" in WiFi sensing. Traditionally, WiFi gesture recognition systems aim to generalize across unseen physical domains—such as new environments, locations, or user orientations, and accordingly, most domain-adversarial methods have been designed based on these physical labels. However, we argue that in the context of WiFi sensing, physical domains do not align with the type of domain differentiation that adversarial learning is designed to address. Specifically, domain adversarial learning seeks to align data distributions in the feature space, not domains that differ primarily in their physical semantics. This technique was originally developed for high-dimensional modalities like vision, where domain labels (e.g., real vs. cartoon images) tend to correspond closely with data distributional shifts. In contrast, WiFi sensing exhibits fundamentally different characteristics, its low-dimensional, and noisy nature often causes physical labels to poorly reflect the true distributional structure of sensing data. As illustrated in Figure 1 , even within the same physical environment, variations in user position, orientation, and unmodeled multipath effects can introduce greater variability in CSI than the environment change itself. As a result, samples within