mmAnomaly: Leveraging Visual Context for Robust Anomaly Detection in the Non-Visual World with mmWave Radar CCS: Computing methodologies Scene anomaly detection CCS: Computing methodologies Activity recognition and understanding CCS: Human-centered computing Ubiquitous and mobile computing

Tarik Reza Toha, Shao-Jung (Louie) Lu, Mahathir Monjur, Shahriar Nirjon · 2026

Read mmAnomaly: Leveraging Visual Context for Robust Anomaly Detection in the Non-Visual World with mmWave Radar CCS: Computing methodologies Scene anomaly detection CCS: Computing methodologies Activity recognition and understanding CCS: Human-centered computing Ubiquitous and mobile computing by Tarik Reza Toha, Shao-Jung (Louie) Lu, Mahathir Monjur, Shahriar Nirjon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-00382.txt.

Introduction

Toha and colleagues work on anomaly detection with millimetre-wave radar. Radar can sense people through clothes and some walls, where cameras fail or intrude on privacy. But radar signals bend with materials, clutter, and stray reflections, so harmless shifts often look like anomalies. Their mmAnomaly system adds RGBD visual context to fix this. A fast classifier reads scene shape and materials, a diffusion model predicts the radar spectrum that scene should give, and a comparison step checks prediction against what the radar sees.

Catalog record

CREATORTarik Reza Toha, Shao-Jung (Louie) Lu, Mahathir Monjur, Shahriar Nirjon
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64481
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.00382

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

mmAnomaly: Leveraging Visual Context for Robust Anomaly Detection in the Non-Visual World with mmWave Radar CCS: Computing methodologies Scene anomaly detection CCS: Computing methodologies Activity recognition and understanding CCS: Human-centered computing Ubiquitous and mobile computing Tarik Reza Toha, Shao-Jung (Louie) Lu, Mahathir Monjur, Shahriar Nirjon Abstract . mmWave radar enables human sensing in non-visual scenarios—e.g., through clothing or certain types of walls—where traditional cameras fail due to occlusion or privacy limitations. However, robust anomaly detection with mmWave remains challenging, as signal reflections are influenced by material properties, clutter, and multipath interference, producing complex, non-Gaussian distortions. Existing methods lack contextual awareness and misclassify benign signal variations as anomalies. We present mmAnomaly, a multi-modal anomaly detection framework that combines mmWave radar with RGBD input to incorporate visual context. Our system extracts semantic cues—such as scene geometry and material properties—using a fast ResNet-based classifier, and uses a conditional latent diffusion model to synthesize the expected mmWave spectrum for the given visual context. A dual-input comparison module then identifies spatial deviations between real and generated spectra to localize anomalies. We evaluate mmAnomaly on two multi-modal datasets across three applications: concealed weapon localization, through-wall intruder localization, and through-wall fall localization. The system achieves up to 94% F1 score and sub-meter localization error, demonstrating robust generalization across clothing, occlusions, and cluttered environments. These results establish mmAnomaly as an accurate and interpretable framework for context-aware anomaly detection in mmWave sensing. 1. Introduction mmWave Anomaly Detection Applications. mmWave anomaly detection has several critical applications, particularly in security ( 11 ; 3 ; 42 ; 48 ) , healthcare ( 14 ; 12 ; 37 ; 46 ) , and industrial monitoring ( 25 ; 36 ; 10 ) , where traditional sensors like cameras, infrared, and acoustic sensors fall short. In security screening, mmWave can penetrate clothing and detect concealed weapons or contraband that optical cameras fail to recognize ( 11 ; 3 ) . In healthcare, mmWave enables non-contact physiological monitoring, such as detecting abnormal respiration patterns indicative of medical conditions like sleep apnea ( 14 ; 12 ) . In industrial settings, mmWave can identify structural anomalies ( 25 ) , material defects ( 36 ) , or equipment malfunctions ( 10 ) in harsh environments where visibility is limited. These applications highlight the unique advantages of mmWave sensing in scenarios where other modalities alone are ineffective. (a) without concealed weapon (b) with concealed weapon Figure 1 . mmAnomaly detects and localizes anomaly (e.g., concealed weapons) by comparing real mmWave data with mmWave signals generated from RGBD images, using the fact that visual appearance stays the same while mmWave reflections change when hidden items are present. Challenges in mmWave Anomaly Detection. The challenges of mmWave anomaly detection stem from the unique properties of mmWave signals, which differ fundamentally from other sensing modalities. Unlike visual or infrared data, mmWave signals exhibit strong sensitivity to environmental conditions, such as surface reflectivity, material properties, and multipath effects. Additionally, mmWave signals do not conform to the typical noise models assumed in traditional anomaly-detection algorithms. Instead, they are subject to complex propagation phenomena, including constructive and destructive interference, non-line-of-sight reflections, and diffraction, which evolve with scene dynamics and introduce structured yet non-stationary distortions. Moreover, anomalies in mmWave spectra rarely appear as isolated spikes or sparse outliers; they are often embedded within subtle waveform deviations or spatially dependent fluctuations that mirror benign environmental variation. These factors create a pronounced mismatch between the physics-driven nature of mmWave signals and the statistical assumptions of conventional models, making it difficult to distinguish true anomalies from context-induced variability without leveraging additional environmental cues. Limitations of Existing Anomaly Detection Methods. Existing anomaly detection methods for mmWave radar fall into two broad categories, each with significant limitations. The first includes vision-inspired models such as GLAD ( 45 ) , GLASS ( 4 ) , and GeneralAD ( 34 ) , which are designed for structured signals like images or time series. When retrofitted for mmWave data, these models fail to capture subtle waveform distortions and structured interference patterns, often misclassifying benign variations as anomalies due to their reliance on simplistic outlier-based assumptions. The second category includes RF-specific models like SiFall ( 18 ) and CGAN ( 39 ) , which are better aligned with wireless signals but still fall short for mmWave. These models assume simplified noise characteristics and/or overlook critical context such as scene geometry, material properties, and multipath effects. As a result, they cannot reliably distinguish true anomalies from context-driven variations, highlighting the need for multimodal approaches that incorporate visual cues. Leveraging Visual Context. While mmWave signals can detect anomalies hidden from visual sensors—such as concealed weapons, intruders behind walls, or falls behind obstructions—relying exclusively on radar data limits anomaly detection performance. mmWave reflections are highly sensitive to environmental and scene-specific factors like surface materials, furniture placement, and dynamic clutter, which can introduce significant variability and ambiguity. Although visual sensors cannot directly observe these hidden anomalies, they provide valuable context about visible environmental structures and scene layouts. By leveraging visual context from a co-located RGBD camera, anomaly detection algorithms can better calibrate their expectations of normal mmWave reflections given the observed materials and spatial arrangements. We conduct a thorough empirical study to understand the effect of incorporating visual context and observe that visual cues consistently enhance detection accuracy and robustness compared to mmWave-only models. This demonstrates that visual guidance plays a critical role in improving the reliability of mmWave anomaly detection systems. We Propose mmAnomaly: A Cross-Modal Generative Approach. To operationalize visual context for mmWave anomaly detection, we introduce mmAnomaly , a multi-modal system that bridges low-level radar signals with high-level scene understanding through cross-modal generation. In this paper, we define an anomaly as a rare mmWave-observable event that cannot be observed by the visible RGBD scene, focusing on two broad classes: (i) unusual physical body movements (e.g., abrupt falls or struggling motions), and (ii) unexpected metallic objects on or near the body (e.g., concealed weapons). Conceptually, mmAnomaly instantiates a new sensing paradigm: instead of relying on radar-only statistical priors, it uses a generative model to synthesize the expected mmWave response from an anchor sensor (the camera). It treats deviations from this prediction as evidence of an anomaly. To our knowledge, mmAnomaly is the first to synthesize expected mmWave signals directly from RGBD inputs of a co-located camera, enabling anomaly detection in scenarios where visual sensors or mmWave alone are insufficient. Trained solely on regular (anomaly-free) data, mmAnomaly learns how visual semantics—such as material properties, human pose, and environmental layout—shape radar reflections. At runtime, mmAnomaly compares the predicted mmWave response with the actual measurement; the resulting residual localizes anomalies in the non-visual domain, such as concealed weapons, intruders behind walls, or falls behind obstructions. We implement this idea by integrating RGBD perception modules with a conditional generative backbone and a residual-based anomaly localizer. By grounding radar interpretation in visual context, mmAnomaly reduces false positives, enhances robustness to environmental variability, and improves generalization across settings. Figure 1 illustrates this principle: even when the RGBD view appears normal (e.g., a person walking naturally), mmAnomaly exploits subtle residuals between observed and predicted mmWave reflections to reveal and localize hidden threats such as concealed weapons. Experimental Evaluation and Results. We

Read the complete text