In-Sensor Motion Recognition with Memristive System and Light Sensing Surfaces
Hritom Das, Imran Fahad, SNB Tushar, Sk Hasibul Alam, Graham Buchanan, Danny Scott, Garrett S. Rose, and Sai Swaminathan · 2025
Read In-Sensor Motion Recognition with Memristive System and Light Sensing Surfaces by Hritom Das, Imran Fahad, SNB Tushar, Sk Hasibul Alam, Graham Buchanan, Danny Scott, Garrett S. Rose, and Sai Swaminathan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06829.txt.
Introduction
Hritom Das and seven co-authors present a 2025 device design that recognizes hand motion inside the sensor itself, at very low power. A light-sensing surface captures gestures as analog signals. A memristive chip with a winner-take-all circuit then classifies the motion on the spot, with no round trip to a processor.
The team validated the system on four swipe gestures: left-to-right, right-to-left, bottom-to-top, and top-to-bottom. Setting the weights cost about 4.17 nanojoules, and each test cost under one nanojoule. Accuracy reached 97.22 percent, holding up under 5 percent noise. Because it sips power, the design could run on harvested solar energy. Local processing also keeps the data private.
Catalog record
| CREATOR | Hritom Das, Imran Fahad, SNB Tushar, Sk Hasibul Alam, Graham Buchanan, Danny Scott, Garrett S. Rose, and Sai Swaminathan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-06829
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
In-Sensor Motion Recognition with Memristive System and Light Sensing Surfaces Hritom Das, Imran Fahad, SNB Tushar, Sk Hasibul Alam, Graham Buchanan, Danny Scott, Garrett S. Rose, and Sai Swaminathan Abstract In this paper, we introduce a novel device architecture that merges memristive devices with light-sensing surfaces, for energy-efficient motion recognition at the edge. Our light-sensing surface captures motion data through in-sensor computation. This data is then processed using a memristive system equipped with a HfO2-based synaptic device, coupled with a winner-take-all (WTA) circuit, tailored for low-power motion classification tasks. We validate our end-to-end system using four distinct human hand gestures—left-to-right, right-to-left, bottom-to-top, and top-to-bottom movements—to assess energy efficiency and classification robustness. Our experiments show that the system requires an average of only 4.17 nJ 4.17\text{\,}\mathrm{nJ} for taking our processed analog signal and mapping weights onto our memristive system and 0.952 nJ 0.952\text{\,}\mathrm{nJ} for testing per movement class, achieving 97.22% accuracy even under 5% noise interference. A key advantage of our proposed architecture is its low energy requirement, enabling the integration of energy-harvesting solutions such as solar power for sustainable autonomous operation. Additionally, our approach enhances data privacy by processing data locally, reducing the need for external data transmission and storage. I Introduction For decades, smart environments have been heralded as transformative, enhancing understanding of context, actions, and activities in various settings such as public areas, offices, homes [ 1 ] , workshops, factories, and healthcare centers [ 2 ] . The realization of this vision hinges on the development of autonomous optical sensing technologies, such as light-sensing surfaces, which surpass traditional camera-based methods and von Neumann architectures in energy efficiency, scalability, and privacy preservation. Breakthroughs in these light-sensing surfaces [ 3 , 4 ] have enabled the integration of sensing and computation into everyday objects, enabling applications from gesture-based inputs to non-contact activity detection. Yet, efficiently processing the voluminous data from these sensors while maintaining low power consumption and latency remains a challenge. A promising approach to mitigate these challenges is in-sensor computation [ 5 ] , which entails processing sensory data in the analog domain before it undergoes analog-to-digital conversion (ADC) and digital computation. This method compresses high-dimensional data from extensive sensor arrays into manageable forms, reducing redundancy [ 6 , 7 ] and addressing privacy concerns by avoiding the transmission of raw data. Such recent advances in sensor computation have enabled computational light-sensing surfaces [ 8 ] that process raw data into essential vision features such as position, orientation, direction, speed, and identification within the analog domain. However, despite advances in computational light sensing surfaces, energy demands exist for wireless data transmission in intelligent motion detection systems, necessitating frequent battery changes. Integrating memristive synaptic devices with computational light sensors offers a promising solution for additional energy savings. This may allow for the classification of motion sensor data directly on the device, avoiding the need for power-intensive wireless transmission of raw sensor data. Therefore, in this work, we propose integrating computational light-sensing surfaces with a memristive system and characterize its performance for power and motion recognition tasks. To the best of our knowledge, this work is the first instance of integrating computational light-sensing surfaces with a memristive system. We develop our computational light-sensing surfaces utilizing a heterogeneous array of photovoltaic (PV) cells Fig. 1 . When an object moves on top of our sensing surface, we achieve in-sensor analog computation by combining the weighted output of PV cells, producing time-resolved differential voltage measurement data. The data is processed by our memeristive system, which we term as Memristive Processing Unit (MPU). Our MPU is designed utilizing \text{HfO}{\vphantom{\text{X}}}_{\smash[t]{\text{2}}} -based synaptic devices [ 9 ] for an energy-efficient design. We use a simple synaptic crossbar capable of in-sensor computation as a dot-product engine (DPE) [ 10 , 11 ] . Once data is processed by the crossbar, we use a winner-take-all (WTA) circuitry as a simple ranking circuit based on the maximum current from the columns (from the synaptic crossbar), which can be utilized for motion classification tasks with a low power budget. Consequently, the MPU is employed not only to train the synaptic crossbar but also to test incoming sensory data in real-time, all within a constrained energy budget. To evaluate the robustness of our design, we introduce uniformly random noises to impact the column current of the system. The key contributions of this paper are as follows. 1. An end-to-end memristive system is proposed for analog signal mapping and testing object motion data. 2. Current-controlled synaptic elements are used to construct a reliable and low-power memristive system. 3. An evaluation of the proposed system for object motion recognition with real-world distinct motion classes 4. An empirical validation of classification robustness by testing against various random noises injected in the column current. Overall, our system paves the way for a novel approach to energy-efficient motion recognition at the edge. Fig. 1: Shows a computational light sensing surface (a) with spatially distributed photovoltaic cells (b) producing a motion-incited analog output for four motions: (c) top to bottom motion (d) left to right motion (e) right to left motion (f) bottom to top motion. The data is then pre-processed and fed to a memristive system for classification and prediction I-A Background on Light Sensing Surfaces Historically, light-sensitive surfaces have been created using a network of PV cells and photodiodes, which function as sensors to detect object movement. Objects moving across these surfaces block light to each cell, altering the photocurrent and voltage characteristics. Specifically, PV cells in light exhibit three critical properties: photocurrent ( I_{PD} ), dark current ( I_{D} , also known as leakage current), and diode capacitance ( C_{D} ). The overall output current ( I_{OUT} ) merges I_{PD} and I_{D} , and is converted to a voltage ( V_{PD} ) using a load resistance ( R_{L} ). In detecting object motion via light sensing, previous systems have used PV cells in two primary modes: a) Photoconductive mode and b) Photovoltaic mode. In photoconductive mode, an external reverse bias voltage aligns V_{PD} directly with light intensity, while reducing diode capacitance C_{D} , improving response time. Thus, changes in V_{PD} due to light blockage by objects can be detected. This approach is widely utilized, as in Ma et al.’s SolarGest [ 12 ] , which uses PV cells for hand motion detection. Though highly sensitive, it requires external power. In contrast, the photovoltaic mode functions independently of external power. It relies on the reduced power generation by a photodiode ( V_{PD} ) when light is partially obstructed by an object. Monitoring this change in the photodiode’s output power allows for detecting obstructions by near-field objects. This method is effectively used in finger motion recognition, as shown by Li et al. [ 13 ] in their self-powered system, and in ambient light sensing surfaces for activity detection by Zhang et al. [ 14 ] . While effective for gathering raw data on object motion, these techniques face challenges, particularly in data transfer to memory for processing, which aligns with the von Neumann architecture. This transfer can lead to power and latency issues. To address this, our design (Fig. 1 (a)), inspired by Zhang et al.’s proposal [ 15 ] to process analog data