COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detection
Darryl Cherian Jacob, Xinyu Liu, Kai Wang, Pan He · 2026
Read COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detection by Darryl Cherian Jacob, Xinyu Liu, Kai Wang, Pan He (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15325.txt.
Introduction
Darryl Cherian Jacob and three others present COPRA, a conditional parameter adaptation framework for video anomaly detection with vision-language models. Old methods suffer a training-inference mismatch: static tuning under distribution shift, and sparse training frames versus dense run-time segments. COPRA instead makes input-own knob updates that live adapt a frozen model per video segment in both training and run time. It beats static baselines in-domain and cross-domain, and extends to video Q&A and dense labels.
Catalog record
| CREATOR | Darryl Cherian Jacob, Xinyu Liu, Kai Wang, Pan He |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46696 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.15325
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detection Darryl Cherian Jacob, Xinyu Liu, Kai Wang, Pan He Abstract Vision-language models (VLMs) have demonstrated impressive performance in video anomaly detection (VAD), while providing interpretable predictions. However, we identify a key yet underexplored VAD issue: the mismatch between training and inference in both data distribution and model configuration. First, many VLM-based VAD approaches rely on a static parameter paradigm after training (or post-adaptation), causing them to overfit the training distribution and limiting generalization to unseen domains. Consequently, these methods lack effective test-time adaptation under substantial distribution shifts, such as novel environments or previously unseen anomaly types. Second, they treat VLMs as reasoning-based classifiers trained on extremely sparse frames from long videos, but perform inference on densely sampled short segments via sliding windows, leading to inherent inconsistencies between training and testing. To address these limitations, we propose COPRA, a conditional parameter adaptation framework for VLM-based VAD. Rather than relying on static post-training adaptation (e.g., fixed prompts or shared parameter updates), COPRA introduces instance-conditioned parameter adaptation, where a lightweight generator predicts input-specific parameter updates to dynamically modulate a frozen VLM on the fly, enabling per-segment adaptation during both training and inference. Empirically, our method achieves strong performance on standard VAD benchmarks. More broadly, it consistently outperforms static baselines in both in-domain and cross-domain settings, and further generalizes beyond VAD to unseen tasks such as multiple-choice Video Question Answering and Dense Captioning. These results suggest that COPRA provides an effective weight space generation mechanism for foundation models, enabling more scalable, adaptive, and context-aware video understanding. The code will be released at https://github.com/THE-MALT-LAB/COPRA 1 Introduction Video anomaly detection (VAD) seeks to identify and localize rare and unexpected events in long videos, often under significant variability in scene context, camera viewpoints, and anomaly types. Recent VAD approaches based on vision-language models (VLMs) 49 ; 50 ; 47 ; 55 are particularly promising, via leveraging rich semantic priors from large-scale pretraining and enabling interpretable predictions. Figure 1: Illustration of the generalization gap in VAD caused by training–inference mismatch and train–test distribution shift. Existing methods rely on shared, static parameters, forcing diverse video patterns into a single compromise solution. As a result, the decision boundary learned from the training dataset often fails to generalize to unseen test distributions, causing some samples to fall on the wrong side of the boundary . We address this limitation through instance-conditioned parameter adaptation, where each video dynamically pulls the decision landscape toward a more suitable boundary via input-specific parameter updates , improving robustness and generalization. However, effectively adapting VLMs to VAD remains an actively evolving and largely underexplored challenge. As illustrated in Figure 1 , existing VAD approaches 49 ; 50 ; 47 ; 13 ; 55 ; 56 typically learn a single, shared parameter space, despite the substantial heterogeneity across anomaly types, temporal dynamics, and scene contexts. During training, each heterogeneous sample induces a distinct gradient direction that may conflict with others, leading to parameters that converge to a compromise rather than being optimal for any single sample 34 . Such static, shared adaptations often overfit to dataset-specific patterns encoded in prompts 47 or model parameters 50 , rather than capturing transferable notions of abnormality. This leads to a pronounced generalization gap, which we term the shared parameter bottleneck . For example, detecting a traffic accident 16 , physical assault 33 , or suspicious retail behavior requires distinct visual cues, temporal dynamics, and semantic priors. A single shared adaptation must reconcile these heterogeneous demands, leading to degraded robustness and generalization. Moreover, prevailing VAD pipelines exhibit a fundamental training–inference mismatch: models are trained on sparsely sampled frames from long videos with weak video-level supervision, but deployed on densely sampled sliding windows for fine-grained localization. As a result, parameters learned from sparse inputs are applied to dense local decisions, degrading performance under variations in viewpoints, environments, motion patterns, and anomaly semantics. In this paper, we advocate an instance-specific perspective on VAD 34 ; 11 . Rather than learning a single shared parameter set, we infer model parameters on the fly for each video segment via instance-conditioned adaptation, where most parameters are shared and frozen, and only a small subset is dynamically customized to emphasize input-specific behaviors. This is motivated by the diversity of anomaly scenarios: some events (e.g., explosions, fires, collisions) can be identified from a few salient frames, while others (e.g., shoplifting, burglary, loitering) require modeling subtle motion patterns and temporal interactions. Consequently, reliable detection spans cues from coarse appearance to fine-grained temporal reasoning, making existing VAD approaches based on a shared-parameter paradigm inherently a compromise. A promising direction is to enable on-the-fly, input-conditioned adaptation that tailors model behavior to each video input. Accordingly, we propose COPRA, a conditional parameter adaptation framework for VLM-based VAD that predicts input-specific parameter updates for each video, dynamically modulating a largely frozen VLM on the fly. In practice, COPRA produces instance-conditioned LoRA weights for customized per-input adaptation. As illustrated in Figure 1 , COPRA addresses the limitation of existing static adaptation approaches, where the decision boundary learned from the training dataset often fails to generalize to unseen test distributions , by dynamically pulling the decision landscape toward a more suitable boundary for each input, thereby improving robustness and generalization under distribution shift. We further employ reinforcement learning fine-tuning to guide token-level parameter generation under video-level supervision and explore flexible frame sampling strategies. Our main contributions can be summarized as follows: 1. We identify a fundamental limitation of existing VAD approaches with the shared parameter bottleneck , where static, shared adaptations force heterogeneous requirements into a single compromised solution, and advocate a shift toward on-the-fly, input-conditioned adaptation. 2. We present COPRA, the first framework to formulate VAD as an instance-conditioned adaptation problem, where a VLM is dynamically specialized for each video segment. 3. We introduce a conditional parameter generation framework for VLMs that enables dynamic, per-input adaptation through generated LoRA weights. We further incorporate reinforcement learning fine-tuning under video-level supervision and investigate scenario-aware frame sampling strategies. Our framework alleviates training–inference mismatch and reduces cross-domain interference, improving robustness across diverse anomaly patterns. 4. We demonstrate strong performance and robust generalization across standard VAD benchmarks, consistently outperforming static adaptation baselines in both in-domain and cross-dataset settings. Moreover, the learned adaptation transfers effectively beyond VAD to downstream video understanding tasks. 2 Related Work VLMs for VAD . Recent works leverage VLMs for their strong cross-modal reasoning capabilities 27 ; 22 . Existing approaches largely treat VLMs as fixed operators: some attach task-specific detection heads to frozen backbones 4 , others decouple captioning and reasoning via VLM–LLM pipelines 49 ; 44 , while prompt-based methods (e.g., VERA 47 ) or post-hoc frame selection strategies 51 ; 50 avoid updating model parameters. Shared Parameter Bottleneck in VLM-based VAD. As previously observed, existing VLM-based VAD approaches adapt models through shared parameters, either by optimizing global prompts 47 or fine-tuning model weights 50 . This introduces a fundamental limitation: model behavior is governed by training data statistics and a fixed operator, hindering generalization to novel videos. As a result, performance degrades on unseen data. Consistent with model editing studies 34 ; 11 , a single shared update is inherently driven toward a