OrganicHAR: Towards Activity Discovery in Organic Settings for Privacy Preserving Sensors Using Efficient Video Analysis Journal: IMWUT Volume: 9 4 203 12 DOI: 10.1145/3770674 CCS: Human-centered computing Ubiquitous and mobile computing systems and tools CCS: Computing methodologies Planning for deterministic actions CCS: Human-centered computing Collaborative interaction CCS: Human-centered computing Interactive systems and tools CCS: Human-centered computing Ambient intelligence
Prasoon Patidar, Riku Arakawa, Ricardo Graça, Rúben Moutinho, Adriano Soares, Ana Vasconcelos, Filippo Talami, Joana Couto da Silva, Inês Silva, Cristina Mendes Santos, Mayank Goel, Yuvraj Agarwal · 2026
Read OrganicHAR: Towards Activity Discovery in Organic Settings for Privacy Preserving Sensors Using Efficient Video Analysis Journal: IMWUT Volume: 9 4 203 12 DOI: 10.1145/3770674 CCS: Human-centered computing Ubiquitous and mobile computing systems and tools CCS: Computing methodologies Planning for deterministic actions CCS: Human-centered computing Collaborative interaction CCS: Human-centered computing Interactive systems and tools CCS: Human-centered computing Ambient intelligence by Prasoon Patidar, Riku Arakawa, Ricardo Graça, Rúben Moutinho, Adriano Soares, Ana Vasconcelos, Filippo Talami, Joana Couto da Silva, Inês Silva, Cristina Mendes Santos, Mayank Goel, Yuvraj Agarwal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18455.txt.
Introduction
Prasoon Patidar and eleven co-authors present OrganicHAR, a 2026 activity discovery system that centers sensor capabilities instead of forcing privacy-preserving sensors to mimic camera labels. It finds natural signal patterns from radar, lidar, and thermal sensors, calling vision-language models only at key moments for scene understanding. With twelve participants it reaches 79 percent accuracy on coarse activities and 73 percent on fine-grained ones, while cutting vision-model queries by 90 percent.
Catalog record
| CREATOR | Prasoon Patidar, Riku Arakawa, Ricardo Graça, Rúben Moutinho, Adriano Soares, Ana Vasconcelos, Filippo Talami, Joana Couto da Silva, Inês Silva, Cristina Mendes Santos, Mayank Goel, Yuvraj Agarwal |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 117996 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.18455
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
OrganicHAR: Towards Activity Discovery in Organic Settings for Privacy Preserving Sensors Using Efficient Video Analysis Journal: IMWUT Volume: 9 4 203 12 DOI: 10.1145/3770674 CCS: Human-centered computing Ubiquitous and mobile computing systems and tools CCS: Computing methodologies Planning for deterministic actions CCS: Human-centered computing Collaborative interaction CCS: Human-centered computing Interactive systems and tools CCS: Human-centered computing Ambient intelligence Prasoon Patidar, Riku Arakawa, Ricardo Graça, Rúben Moutinho, Adriano Soares, Ana Vasconcelos, Filippo Talami, Joana Couto da Silva, Inês Silva, Cristina Mendes Santos, Mayank Goel, Yuvraj Agarwal Abstract . Deploying human activity recognition (HAR) at home is still rare because sensor signals vary wildly across houses, people, and time, essentially requiring in-situ data collection and training. Prior approaches use cameras to generate training labels for privacy-preserving sensors (LiDAR, RADAR, Thermal), but this forces sensors to detect predefined activities that cameras can see yet the sensors themselves cannot reliably distinguish. In this work, we introduce OrganicHAR, an activity discovery framework that inverts this relationship by placing sensor capabilities at the center of activity discovery. Our approach identifies naturally occurring signal patterns using privacy-preserving sensors, leverages Vision Language Models (VLMs) only during these key moments for scene understanding, and discovers discrete activity labels at granularities that these sensors can reliably detect. Our evaluation with 12 participants demonstrates OrganicHAR’s effectiveness: it achieves 79% accuracy for coarse (4-5) activities using only basic ambient sensors (radar, lidar, thermal arrays), and 73% accuracy for fine-grained (8-9) activities when a wearable IMU, depth, and pose sensor are added. OrganicHAR maintains 77% accuracy on average across configurations while discovering 4-8 categories per user (15 across all users) tailored to each environment and sensor capabilities. By triggering video processing only at key moments identified by local sensors, we reduce queries to VLM by 90%, enabling practical and privacy-preserving activity recognition in natural settings. † † cc-license: by 1. INTRODUCTION Smart homes that can infer the activities of their occupants and offer them insights, automation, and assistance without privacy-invasive cameras or microphones have been a longstanding vision of the research community and companies ( 61 ; 16 ; 17 ) . Yet despite significant advances in Human Activity Recognition (HAR) techniques, their deployment remains largely unrealized in most households ( 69 ; 65 ) . While these HAR systems achieve impressive performance using benchmarks and in controlled settings, they falter in real home settings. This persistent disconnect stems not from technological limitations alone, but from a fundamental misalignment in how we have conceptualized the problem: we face the dual challenge of (i) deploying models in organic settings where users perform diverse, evolving activities in countless variations and (ii) simultaneously obtaining high-quality training labels to identify what activities are occurring in these complex, unstructured environments. Most existing HAR approaches start with a set of activities to detect and classify. For instance, Patidar et al. ( 56 ) trained privacy-preserving sensors ( e.g. , radar, lidar) to detect 17 common activities of daily living at home. While these approaches prove effective for short durations, the assumption that the same set of activities happens with similar consistencies across all environments breaks down in the long term, as users engage in everyday activities in their very own idiosyncratic ways ( 30 ; 65 ) . This reliance on pre-defined activity sets creates challenges in aligning with activities that matter to users and what activities these sensors can reliably detect in diverse environments. Rather than imposing a fixed set of activities across all environments, we need HAR systems that can autonomously discover what activities are reliably detectable given the specific sensors deployed in each unique setting. This discovery capability is crucial for practical adoption—users must understand their sensors’ actual detection capabilities before making informed decisions about how they want to use these systems. In this paper, we introduce OrganicHAR 1 1 1 https://github.com/synergylabs/OrganicHAR (Figure 1 ), a novel framework that takes the first step towards this vision by discovering activities that naturally emerge from available sensing capabilities, rather than imposing predefined categories. We start by identifying potentially meaningful patterns ( i.e. , recurring spatial patterns and temporal fluctuations) within sensor data. Next, we selectively leverage Vision Language Models ( 15 ) (VLMs) to understand what activities these patterns represent, and convert descriptions (in natural language) into discrete labels to train HAR models. Our approach offers two advantages critical for practical deployment: (1) it allows us to control the granularity of recognized activities for different environments and available sensing capabilities; and (2) it reduces computational overhead by processing only essential video segments during training. Developing the OrganicHAR framework required solving two major technical challenges. First, we developed novel techniques to identify meaningful interaction moments from multimodal time-series sensor data, without requiring prior activity models. Second, to address the inconsistent activity descriptions generated by VLMs, we implemented a clustering method that transforms these variable VLM descriptions into consistent, sensor-appropriate activity labels—effectively connecting rich semantic understanding of VLMs with the limited capabilities of privacy-preserving sensors. Figure 1. The OrganicHAR framework discovers activity labels through a three-step sensor-first approach. Step 1: Privacy-preserving sensors identify “key moments” with recurring patterns (clusters) or anomalies (peaks). Step 2: A vision language model (VLM) generates natural language descriptions only for these key moments. Step 3: Descriptions are converted into activity labels with adaptive granularity tuned for downstream applications, following human-intuitive hierarchies where specific activities ( e.g. , "Sandwich Prep") naturally group into broader categories ("Food Prep" → "Counter Activity") while maintaining semantic meaning. Unlike prior approaches that use predefined activities and continuous video monitoring (top right), OrganicHAR discovers activities organically from sensor capabilities while reducing video processing by 90%. Three-step workflow showing sensor-first activity discovery with adaptive granularity Diagram showing OrganicHAR's three-step workflow for activity discovery in a kitchen environment. Left side shows a kitchen with person preparing food, monitored by privacy-preserving sensors (radar, lidar, thermal) and a camera used only during training phase. Center section illustrates Step 1 with two visual panels: top panel shows colored clusters representing recurring sensor signal patterns, bottom panel displays a time-series graph with orange dots marking temporal anomalies above a threshold line. These key moments feed into Step 2, where GPT-4o vision language model generates natural language descriptions like "Person spreading peanut butter on bread slice" and "Individual pouring milk over cereal in bowl." Step 3 shows hierarchical activity label organization at three granularity levels (Relaxed, Balanced, Conservative) with orange boxes representing discovered labels: specific activities like "Sandwich Prep" and "Cereal Prep" merge into "Food Prep," which further merges into "Counter Activity" at coarser levels. Similar hierarchies shown for Fridge Access, Cleaning Counter, Coffee Prep, and Dish/Hand Wash activities grouped by location (Food Prep, Cleaning, Drink Prep, Sink Activity). Top right corner contains comparison table contrasting Prior Approaches (pre-defined activity sets, fixed granularity, continuous video) versus OrganicHAR's sensor-first approach (discovered activities, adaptive granularity, key moments only). To build and evaluate the performance of OrganicHAR, we collected a comprehensive multimodal dataset (comprising a Doppler RADAR, a 2D LiDAR, a thermal array, a wearable IMU, pose and depth sensors) from 12 participants, who performed four different breakfast preparation tasks (preparing sandwiches, tea, coffee, and cereal). Each participant performed these tasks naturally without instructions, allowing us to capture organic activity patterns