Beyond the Beep: Scalable Collision Anticipation and Real-Time Explainability with BADAS-2.0

Roni Goldshmidt, Hamish Scott, Lorenzo Niccolini, Hernan Matzner · 2026

Read Beyond the Beep: Scalable Collision Anticipation and Real-Time Explainability with BADAS-2.0 by Roni Goldshmidt, Hamish Scott, Lorenzo Niccolini, Hernan Matzner (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05767.txt.

Introduction

Goldshmidt and three colleagues advance collision anticipation with BADAS-2.0, built on fine-tuned V-JEPA2 over dashcam data. A 10-group long-tail benchmark grows labeled video from 40k to 178,500 clips. Distillation yields compact edge models 7 to 12 times faster at near-parity accuracy. Heatmaps plus a vision-language reasoner explain predictions in real time.

Catalog record

CREATORRoni Goldshmidt, Hamish Scott, Lorenzo Niccolini, Hernan Matzner
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33296
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.05767

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beyond the Beep: Scalable Collision Anticipation and Real-Time Explainability with BADAS-2.0 Roni Goldshmidt, Hamish Scott, Lorenzo Niccolini, Hernan Matzner Abstract We present BADAS-2.0, the second generation of our collision anticipation system, building on BADAS-1.0 7 , which established that fine-tuning V-JEPA2 1 on large-scale, ego-relevant dashcam data dramatically outperforms both academic baselines and ADAS systems. BADAS-2.0 advances the state of the art along three coherent axes. (i) Long-tail benchmark and accuracy: We introduce a new 10-group long-tail benchmark targeting rare and safety-critical driving scenarios — the primary testbed for evaluating model robustness on underrepresented collision types. To populate it, we employ an pipeline in which BADAS-1.0 serves as a deployed active oracle, continuously scoring millions of unlabeled drives to surface high-risk and informative candidates for annotation. Combined with Nexar’s geospatial Atlas platform 13 for targeted harvesting of rare long-tail scenarios, this strategy grows the training corpus from 40k to 178,500 labeled videos ( ∼ \sim 2M clips ), achieving consistent state-of-the-art gains across all benchmark subgroups, with the most pronounced improvements on the hardest long-tail categories. (ii) Knowledge distillation to edge: Domain-specific self-supervised pre-training on 2.25M unlabeled driving videos enables effective distillation into BADAS-2.0-Flash (86M) and BADAS-2.0-Flash-Lite (22M) — delivering a 7–12 × \times runtime speedup while retaining near-parity accuracy, making real-time collision anticipation practical for edge deployment. (iii) Explainability: BADAS-2.0 produces real-time, object-focused attention heatmaps that localize which scene elements drive each prediction. BADAS-Reason 17 extends this into language: a fine-tuned vision-language model that ingests the last video frame together with its heatmap and generates the required driver action alongside structured textual reasoning for avoiding the detected hazard. Inference code and evaluation benchmarks are publicly available. 1 Introduction Ego-centric collision anticipation — predicting whether the filming vehicle will be involved in a collision — is a distinct and harder task than general accident detection. BADAS-1.0 7 established this distinction: it re-annotated public benchmarks (DAD 3 , DADA-2000 6 , DoTA 18 ) to isolate ego-relevant events, and demonstrated that fine-tuning V-JEPA2 1 on 40k ego-centric dashcam clips outperforms both RNN- and transformer-based accident detectors 16 ; 10 and non-commercial forward-collision warning systems 12 . The key insight behind this result is that V-JEPA2’s masked spatiotemporal prediction objective learns to represent the dynamics of an unfolding scene — capturing the motion patterns that precede a collision — whereas prior methods rely on appearance-based features or semantic priors that transfer poorly to the ego-centric dashcam distribution. BADAS-2.0 addresses three unsolved constraints from BADAS-1.0: the training corpus (40k clips) lacked rare long-tail scenarios, leading to fragile performance; inference cost (2.5 s/window) precluded on-device deployment; and the scalar risk score provided no explanation of which object triggered an alert. We address all three through an integrated arc: collect more data, intelligently ; train a stronger model; distill to edge latency; and add explainability via heatmaps and VLM fine-tuning. Each stage builds on the previous: the dataset is assembled using BADAS-1.0 as an oracle with a human-in-the-loop (HITL) annotation process; the distilled models start from SSL weights pre-trained on millions of unlabeled driving videos; and the VLM pipeline is trained on frames selected by the BADAS attention mechanism. Contributions. • Intelligent data mining at scale (Sec. 3 ). We use BADAS-1.0 as an active mining oracle over millions of unlabeled Nexar drives, supplemented by targeted geospatial queries via Nexar Atlas for rare scenarios. The training corpus expands 5 \times to 178,500 labeled videos ( \sim 2M windowed training clips), with a new 10-group long-tail benchmark (888 manually verified clips across animal, cyclist, fog, infrastructure, intersection, motorcyclist, passing/overtaking, pedestrian, rain, and snow). • Improved accuracy across all benchmarks (Sec. 4 ). BADAS-2.0 (ViT-L, 300M) improves consistently over BADAS-1.0 on every benchmark, including Kaggle mAP from 0.925 \to 0.940 , with gains across all long-tail scenario groups. • Knowledge distillation to edge models (Sec. 5 ). We demonstrate that domain-specific SSL pre-training on 2.25M unlabeled driving videos transforms a ViT-S from near-random to near-production quality before any distillation. Subsequent KD from the ViT-L teacher yields BADAS-2.0-Flash (86M, 4.8 ms) and BADAS-2.0-Flash-Lite (22M, 2.8 ms), both within the real-time budget on all tested platforms. • BADAS-Reason: Visual and Textual Explainability (Sec. 6 ). BADAS-Reason extends BADAS with a two-pronged explainability layer. On the visual side, attention maps are extracted at inference time to highlight the objects and regions that triggered each alert. On the textual side, we fine-tune Qwen3-VL-4B 17 via QLoRA 4 on a curated dataset of peak-risk frames paired with manually annotated descriptions, each specifying the recommended driver action and the model’s reasoning behind the alert. The resulting model reduces perplexity by 87% and achieves a 3.6\times improvement in action-match accuracy over the zero-shot baseline. 2 Related Work Collision anticipation. Existing methods range from RNN-based adaptive loss functions 16 to spatial-temporal attention transformers 10 and graph networks modelling agent relationships 5 . BADAS-1.0 7 showed the primary bottleneck is data alignment — ego-relevant events rather than general accident footage — rather than architecture. Active learning and model-assisted mining. Self-training pipelines that use model scores to prioritise unlabeled data for annotation are established in image classification 11 . We apply this to collision anticipation: BADAS-1.0’s risk scores surface high-risk clips for human review, scaling the corpus without exhaustive search. Knowledge distillation for video. Feature- and response-based distillation 9 are well-established. For video, domain-specific SSL pre-training via V-JEPA-style masked prediction 2 ; 1 is a prerequisite: V-JEPA2 was released only at ViT-L scale, and the domain gap from hand-gesture alternatives is too large for supervised fine-tuning alone. VLMs for driving. Large VLMs are increasingly applied to driving 15 . BADAS-Reason uses a fine-tuned VLM as an explanation generator consuming BADAS attention outputs, adding natural-language reasoning at real-time speed. 3 BADAS-2.0 Dataset 3.1 Intelligent Data Collection BADAS-1.0 was trained on \approx 40k clips. Its primary weakness was long-tail coverage: animal crossings, nighttime intersections, and adverse weather were underrepresented, leading to fragile recall on these groups. BADAS-2.0 assembles 178,500 labeled videos through two strategies executed after BADAS-1.0 was trained. Active mining with BADAS-1.0 (Oracle + HITL). We run BADAS-1.0 over millions of unlabeled Nexar drives; clips exceeding a risk threshold are surfaced for human review. Professional annotators independently confirm or reject each label — the oracle acts purely as a retrieval filter, and every clip entering training carries a human-verified label. High-confidence false positives are retained as hard negatives to directly reduce false alarm rates. Rare-event harvesting via Nexar Atlas 13 . Atlas is Nexar’s geospatial intelligence platform, which indexes billions of road miles by scene semantics, weather, road type, time of day, and detected objects. Targeted queries extracted clips matching underrepresented categories — animal crossings, nighttime intersections, rain, snow, fog, passing/overtaking scenarios — and sent them for expert annotation. This directly addresses the long-tail gaps measured in BADAS-1.0. 3.2 Annotation and Labeling Professional annotators applied a seven-type ontology (collision, near-collision, hard brake, normal driving, intersection, passing/overtaking, other), recording direction, third-party involvement, and event timestamps. 3.3 Dataset Statistics Table 1 summarises the three data generations. Table 1: BADAS dataset evolution. Clips are windowed training segments extracted from labeled videos. BADAS-Open BADAS-1.0 BADAS-2.0 Labeled videos 1.5k 40k 178.5k Training clips 15k 400k \sim 2M Positives 0.8k 20k 24.1k Negatives 0.8k

Read the complete text