PROBE : Self-Supervised Visual Prompting for Cross-Domain Road Damage Detection
Xi Xiao, Zhuxuanzi Wang, Mingqiao Mo, Chen Liu, Chenrui Ma, Yanshu Li, Smita Krishnaswamy, Xiao Wang, Tianyang Wang, [2mm] Corresponding authors: [email protected], [email protected] · 2025
Read PROBE : Self-Supervised Visual Prompting for Cross-Domain Road Damage Detection by Xi Xiao, Zhuxuanzi Wang, Mingqiao Mo, Chen Liu, Chenrui Ma, Yanshu Li, Smita Krishnaswamy, Xiao Wang, Tianyang Wang, [2mm] Corresponding authors: [email protected], [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-12410.txt.
Introduction
Xi Xiao and eight co-authors propose a self-supervised method to detect road damage in new places without fresh labeled data. Supervised detectors work well where they were trained, but pavement surveys in a new environment usually demand costly re-annotation. Ordinary self-supervised features are generic and still falter when the domain shifts.
Their framework visually probes the target domain with prompts derived from its own unlabeled images, guiding a frozen vision transformer backbone. An alignment objective then pulls prompt-conditioned source and target representations together. Across four hard benchmarks the method beats supervised, self-supervised and adaptation baselines, with strong zero-shot transfer and good results even when only few labeled examples are added.
Catalog record
| CREATOR | Xi Xiao, Zhuxuanzi Wang, Mingqiao Mo, Chen Liu, Chenrui Ma, Yanshu Li, Smita Krishnaswamy, Xiao Wang, Tianyang Wang, [2mm] Corresponding authors: [email protected], [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-12410
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PROBE : Self-Supervised Visual Prompting for Cross-Domain Road Damage Detection Xi Xiao, Zhuxuanzi Wang, Mingqiao Mo, Chen Liu, Chenrui Ma, Yanshu Li, Smita Krishnaswamy, Xiao Wang, Tianyang Wang, [2mm] Corresponding authors: [email protected], [email protected] Abstract The deployment of automated pavement defect detection is often hindered by poor cross-domain generalization. Supervised detectors achieve strong in-domain accuracy but require costly re-annotation for new environments, while standard self-supervised methods capture generic features and remain vulnerable to domain shift. We propose PROBE , a self-supervised framework that visually probes target domains without labels. PROBE introduces a Self-supervised Prompt Enhancement Module (SPEM), which derives defect-aware prompts from unlabeled target data to guide a frozen ViT backbone, and a Domain-Aware Prompt Alignment (DAPA) objective, which aligns prompt-conditioned source and target representations. Experiments on four challenging benchmarks show that PROBE consistently outperforms strong supervised, self-supervised, and adaptation baselines, achieving robust zero-shot transfer, improved resilience to domain variations, and high data efficiency in few-shot adaptation. These results highlight self-supervised prompting as a practical direction for building scalable and adaptive visual inspection systems. Source code is publicly available: https://github.com/xixiaouab/PROBE/tree/main 1 Introduction Automated pavement defect detection is critical for road safety and infrastructure maintenance, as undetected cracks and potholes can escalate into severe hazards and costly repairs. Current practice relies on supervised detectors such as Faster R-CNN and the YOLO family [ 40 , 20 ] . While these models achieve high accuracy within the training domain, they require exhaustive re-annotation to sustain performance in new environments and degrade sharply under domain shifts caused by variations in pavement materials, lighting, or weather [ 13 , 41 ] . This dependence on large-scale labeling severely limits their scalability in real-world deployments. Self-supervised learning (SSL) has emerged as a promising alternative by learning transferable representations from unlabeled data [ 5 , 16 , 6 ] . Although SSL reduces reliance on labeled data, its learned features are often too generic and lack explicit mechanisms for cross-domain alignment, leading to performance drops when applied to specialized tasks like defect detection [ 55 ] . In parallel, Visual Prompt Tuning (VPT) offers a parameter-efficient way to adapt large Vision Transformers (ViTs) by inserting a small set of learnable tokens [ 10 , 19 ] . VPT and its extensions have shown success in supervised settings [ 23 , 14 ] , but their application to domain adaptation remains limited. Prompts are typically optimized with labels and randomly initialized, failing to leverage the semantic structure embedded in unlabeled target-domain data. This gap motivates our key question: Can we design prompts that are self-supervised, domain-aware, and specialized for defect detection? We address this challenge in the context of single-source \rightarrow single-target unsupervised domain adaptation (UDA) for road-damage detection in a closed-set setting. During pre-training, the model has access to unlabeled source and target images, but no target labels. A lightweight detection head is subsequently trained using a small labeled subset of the source domain, with optional few-shot experiments on the target domain reported separately. We present PROBE , a prompt-enhanced self-supervised framework designed for robust cross-domain pavement defect detection. PROBE integrates two complementary modules: (1) a Self-supervised Prompt Enhancement Module (SPEM) that derives visual prototypes from unlabeled target images and projects them into prompts that condition a frozen ViT backbone, and (2) a Domain-Aware Prompt Alignment (DAPA) objective that aligns prompt-enhanced source and target features via a linear-kernel MMD loss. With the backbone frozen, only lightweight modules and the detection head are trained, yielding a practical and efficient pipeline. Our main contributions are as follows: • We propose a target-aware self-supervised prompting mechanism that generates prompts from unlabeled target-domain images, enabling the model to capture fine-grained, defect-specific semantics beyond generic SSL representations. • We introduce a domain-aware alignment strategy (DAPA) that operates in the prompt-enhanced feature space, explicitly reducing domain discrepancy and improving cross-domain generalization. • We demonstrate through extensive experiments that PROBE consistently outperforms strong SSL, UDA, and prompt-tuning baselines across multiple benchmarks, while being highly parameter-efficient with a frozen backbone and lightweight adaptation modules. Taken together, PROBE shows that prompts distilled from unlabeled target data can serve as semantic anchors for both specialization and alignment, paving the way for practical self-supervised adaptation in real-world infrastructure inspection. 2 Related Work 2.1 Supervised Pavement Defect Detection The prevailing paradigm for pavement defect detection has been supervised learning. Early work employed patch-level CNN classifiers [ 57 ] , followed by modern object detection architectures such as Faster R-CNN [ 40 ] , the YOLO family [ 20 ] , and attention-based variants [ 39 , 48 , 29 , 53 , 30 , 25 , 24 , 28 ] . These methods achieve strong accuracy when sufficient annotations are available and have become widely adopted in practice. However, their reliance on domain-specific labels creates a critical bottleneck: models trained in one region often fail to generalize to new environments with different road materials, lighting, or weather. This domain brittleness makes supervised detection difficult to scale without costly and repeated re-annotation [ 13 ] . 2.2 Self-Supervised Representation Learning Self-supervised learning (SSL) provides a powerful alternative by learning transferable representations from unlabeled data. Contrastive and Siamese frameworks such as SimCLR [ 5 ] , MoCo [ 16 ] , and SimSiam [ 6 ] have significantly reduced the dependency on large labeled corpora. Yet, applying generic SSL representations directly to defect detection exposes two persistent shortcomings. First, these representations often lack the fine-grained, task-specific semantics necessary to capture subtle cracks or potholes. Second, standard SSL does not address domain discrepancy, leaving models vulnerable to performance drops under domain shift [ 55 ] . Recent efforts on defect-aware SSL show promise, but explicit mechanisms for cross-domain adaptation remain underexplored. 2.3 Domain Adaptation for Object Detection Unsupervised domain adaptation (UDA) has been widely studied for object detection to improve robustness across distributions. Early approaches used adversarial alignment [ 13 , 41 ] , classifier discrepancy [ 41 ] , or feature-level regularization [ 42 , 17 ] . More recent work has adapted transformers for UDA, such as CDTrans [ 55 ] , and explored source-free adaptation where only the source-trained model is available at adaptation time [ 32 ] . These strategies demonstrate the importance of aligning cross-domain features but often operate on global statistics, which may dilute fine-grained defect cues that are critical in road inspection. 2.4 Visual Prompt Tuning Visual prompt tuning (VPT) [ 19 , 50 , 54 , 51 , 52 , 15 , 58 ] has recently emerged as a parameter-efficient alternative to full fine-tuning of Vision Transformers (ViT) [ 10 ] . By inserting a small number of learnable tokens into a frozen backbone, VPT enables efficient adaptation and has shown strong performance in supervised scenarios [ 23 , 14 ] . However, its application to domain adaptation remains limited. Existing methods optimize prompts using labels and typically initialize them randomly, overlooking the semantic structure in unlabeled target data. To our knowledge, no prior work has explored generating and adapting prompts in a fully self-supervised manner to simultaneously capture defect-specific semantics and enhance cross-domain robustness. This gap motivates our proposed framework, which leverages unlabeled target-domain imagery to derive task-aware prompts and explicitly align them