From Pixels to Privacy: Temporally Consistent Video Anonymization via Token Pruning for Privacy Preserving Action Recognition
Nazia Aslam Abhisek Ray Joakim Bruslund Haurum Lukas Esterle Kamal Nasrollahi · 2026
Read From Pixels to Privacy: Temporally Consistent Video Anonymization via Token Pruning for Privacy Preserving Action Recognition by Nazia Aslam Abhisek Ray Joakim Bruslund Haurum Lukas Esterle Kamal Nasrollahi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26336.txt.
Introduction
Aslam and co-authors propose a way to hide private traits in video while keeping action recognition working. Video models can pick up face identity and other traits, and motion itself can act as a biometric cue. Their method splits action traits from privacy traits inside a vision transformer, using two special tokens. A score then ranks each space-time tubelet, and only the top ones are kept. Tests show the pruning cuts privacy cues while holding on to action cues.
Catalog record
| CREATOR | Nazia Aslam Abhisek Ray Joakim Bruslund Haurum Lukas Esterle Kamal Nasrollahi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39834 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.26336
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Pixels to Privacy: Temporally Consistent Video Anonymization via Token Pruning for Privacy Preserving Action Recognition Nazia Aslam Abhisek Ray Joakim Bruslund Haurum Lukas Esterle Kamal Nasrollahi Abstract Recent advances in large-scale video models have significantly improved video understanding across domains such as surveillance, healthcare, and entertainment. However, these models also amplify privacy risks by encoding sensitive attributes, including facial identity, race, and gender. While image anonymization has been extensively studied, video anonymization remains relatively underexplored, even though modern video models can leverage spatiotemporal motion patterns as biometric identifiers. To address this challenge, we propose a novel attention-driven spatiotemporal video anonymization framework based on systematic disentanglement of utility and privacy features. Our key insight is that attention mechanisms in Vision Transformers (ViTs) can be explicitly structured to separate action-relevant information from privacy-sensitive content. Building on this insight, we introduce two task-specific classification tokens, an action CLS token and a privacy CLS token, that learn complementary representations within a shared Transformer backbone. We contrast their attention distributions to compute a utility–privacy score for each spatiotemporal tubelet, and keep the top- k k tubelets with the highest scores. This selectively prunes tubelets dominated by privacy cues while preserving those most critical for action recognition. Extensive experiments demonstrate that our approach maintains action recognition performance comparable to models trained on raw videos, while substantially reducing privacy leakage. These results indicate that attention-driven spatiotemporal pruning offers an effective and principled solution for privacy-preserving video analytics. Find code HERE . 1 Introduction Video understanding is a fundamental research area in computer vision, covering a wide range of tasks, including human action recognition, anomaly detection, and temporal action localization. The ability to effectively model the spatiotemporal cues is essential for numerous real‑world applications, including patient monitoring, sports analytics, robotics, and intelligent surveillance systems. Recent models have increasingly focused on learning transferable spatiotemporal representations that generalize across various video understanding tasks [ 27 , 28 , 2 ] . These models are trained on large-scale video datasets to capture the appearance and motion patterns. However, these models raise privacy concerns because they can implicitly encode sensitive personal attributes, such as facial features, gender, and race, within their learned representations. The large-scale collection and automated analysis of such data have motivated stringent regulatory frameworks, including the General Data Protection Regulation (GDPR) [ 26 ] and the California Consumer Privacy Act (CCPA) [ 10 ] , which impose strict constraints on the collection and processing of personal information. These regulations necessitate privacy-aware video understanding systems that strike a balance between task performance and compliance requirements. As these models are increasingly used in real-world applications, it becomes crucial to address the privacy risks associated with video representations, particularly without compromising task performance. Despite the increasing use of advanced video representation models, many early privacy-preserving approaches address privacy risks by directly modifying the input video data, rather than the learned representations. A common strategy is to reduce the spatial resolution of video frames through downsampling, which limits the visibility of identifiable details [ 7 , 19 , 24 ] . Another approach involves detection-based anonymization, in which pretrained detectors locate privacy-sensitive areas in the video. These areas are then obscured using techniques such as blurring [ 30 ] or replaced with synthetic content [ 23 ] . While these methods can reduce visual identifiability, they often fail to anonymize motion patterns in video data, which can still lead to identity leakage. At the same time, aggressive spatial modifications disrupt the spatiotemporal structure required for effective video understanding. As a result, such input-level anonymization strategies both provide limited privacy protection and degrade the performance of downstream recognition tasks. More recent work has shifted toward learning-based privacy-preserving methods, where anonymizers are trained in an adversarial manner to suppress private attributes while maintaining task performance [ 8 , 9 , 29 , 1 ] . These approaches have shown improved results compared to input-level anonymization, particularly in removing appearance-based privacy cues. However, many of them still remain vulnerable to privacy leakage through motion patterns, which can encode identity-related information over time. In contrast, STPrivacy [ 17 ] proposes a video anonymization framework that jointly considers appearance and temporal information. It represents videos as spatiotemporal tubelets and applies token sparsification together with embedding-space anonymization under an adversarial objective. While effective, the token selection process in STPrivacy does not explicitly disentangle utility and privacy signals. As a result, the same tubelets may simultaneously contribute to both action recognition and privacy inference, making it difficult to precisely control the privacy–utility trade-off. Motivated by this limitation, we propose an attention-driven spatiotemporal video anonymization framework that explicitly separates utility and privacy objectives. Our method introduces dedicated classification tokens for action recognition and privacy prediction, and leverages their attention responses to assign each tubelet a utility-privacy score. We then retain the top- k tubelets with the highest scores, thereby pruning tubelets dominated by privacy cues while preserving those critical for action recognition. This allows us to achieve competitive action recognition performance while removing data prone to leaking private information. The contributions of the proposed work are summarized as follows: 1. We introduce two task-specific classification tokens, [act_CLS] and [priv_CLS] , with mutual attention masking. Each token attends to all spatiotemporal tubelets while remaining isolated from the others. 2. We propose an attention-driven tubelet scoring function, s_{i}=\alpha_{i}^{\text{act}}-\lambda_{\text{priv}}\cdot\alpha_{i}^{\text{priv}} , where \alpha_{i}^{\text{act}} and \alpha_{i}^{\text{priv}} are the attention weights assigned to tubelet i by the action and privacy tokens, respectively, and s_{i} is the resulting utility–privacy attention score used for top- k tubelet selection. 3. We conduct extensive experiments on publicly available datasets to validate the effectiveness of the proposed framework under both in-dataset and cross-dataset evaluation settings. 2 Related Work The developments in privacy-preserving vision systems can be broadly categorized into two main approaches: 1) Traditional methods and 2) Learning-based methods. Traditional privacy-preserving methods include techniques such as blurring, pixelation, downsampling, masking, and obfuscation [ 6 , 19 , 3 , 13 ] . These approaches aim to reduce privacy attributes while retaining enough visual context for downstream tasks. However, these methods have two significant limitations. First, they require domain knowledge to accurately identify areas of interest. Second, they do not operate as end-to-end processes; instead, they consist of two distinct steps: detecting or segmenting private objects and then modifying or removing them [ 23 ] . Additionally, these methods often lead to significant degradation of image quality, which can hinder downstream tasks such as expression analysis and attribute prediction [ 21 ] . Previous studies have also shown that these anonymization techniques can be partially reversible, raising concerns about their effectiveness in adversarial settings and the risk of privacy leakage [ 20 ] . Therefore, these approaches may not be sufficient for situations that demand both strong privacy protection and high utility in downstream tasks. Learning-based approaches employ an end-to-end training pipeline to optimize anonymization models, allowing for a trade-off between the downstream task performance and the privacy preservation. Wu et al . [ 29 ] introduced an adversarial training framework that uses privacy labels to train the anonymizer. Building on this idea, MaSS [ 5 ] proposed a similar adversarial framework with a composite loss that selectively retains certain attributes instead