DART: A Vision-Language Foundation Model for Comprehensive Rope Condition Monitoring
Anju Rani, Daniel Ortiz-Arroyo, Petar Durdevic · 2026
Read DART: A Vision-Language Foundation Model for Comprehensive Rope Condition Monitoring by Anju Rani, Daniel Ortiz-Arroyo, Petar Durdevic (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04943.txt.
Introduction
Anju Rani, Daniel Ortiz-Arroyo, and Petar Durdevic introduce DART, a vision-language model for judging the condition of synthetic fibre ropes from a single inspection photo. Such ropes serve offshore, maritime, and industrial work, where inspectors need more than a label. They need severity scores, repair advice, anomaly flags, timelines, and reports.
DART joins a large vision transformer with a Llama language model through a severity-aware fusion module. Damage-focused masking, per-class severity gates, and a special training loss teach it both damage type and ordering. Trained once on thousands of images across fourteen damage classes, it then supports many inspection tasks without further fine-tuning.
Catalog record
| CREATOR | Anju Rani, Daniel Ortiz-Arroyo, Petar Durdevic |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.04943
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DART: A Vision-Language Foundation Model for Comprehensive Rope Condition Monitoring Anju Rani, Daniel Ortiz-Arroyo, Petar Durdevic Abstract The condition monitoring (CM) of synthetic fibre ropes (SFRs) used in offshore, maritime, and industrial settings demands more than a classifier: inspectors need continuous severity estimates, maintenance recommendations, anomaly flags, deterioration timelines, and automated reports, all from a single inspection image. We present DART (Damage Assessment via Rope Transformer), a vision-language foundation model that addresses the full rope inspection workflow through a unified multi-task architecture. DART extends the Joint-Embedding Predictive Architecture (JEPA) to the cross-modal domain by coupling a Vision Transformer (ViT-H/14) with Llama-3.2-3B-Instruct via a Severity-Conditioned Cross-Modal Fusion (SC-CMF) module. Three architectural innovations drive the model’s versatility: (1) HD-MASK, a saliency-guided masking strategy that focuses self-supervised reconstruction on damage-dense patches; (2) per-class learnable severity gates that adaptively weight language grounding by damage category; and (3) a Contrastive Damage Disentanglement (CDD) loss that shapes the embedding space to simultaneously encode damage type, severity ordering, and cross-modal semantics. Trained once on 4,270 images spanning 14 fine-grained rope damage classes, the frozen DART backbone supports downstream tasks without any task-specific fine-tuning: damage classification (93.22% accuracy, 91.04% macro-F1, +38.5 pp over a vision-only baseline), continuous severity regression (Spearman ρ = 0.94 \rho=0.94 , within-1-ordinal accuracy 99.6%), few-shot recognition (89.2% macro-F1 at 20 shots), damage progression modelling (91% monotone interpolation), maintenance action recommendation (94.79% macro-F1), automated inspection report generation (93.22% damage accuracy), out-of-distribution anomaly detection (4.76% anomaly rate), and trajectory-based deterioration phenotyping. These results demonstrate that DART functions as a general-purpose CM backbone that goes well beyond classification, providing actionable inspection intelligence from a single shared representation. Keywords Vision-Language Model \cdot Condition monitoring \cdot JEPA \cdot Self-supervised Learning \cdot Defect detection \cdot Computer vision 1 Introduction Synthetic fibre ropes (SFRs) are critical load-bearing components in offshore lifting operations, mooring systems, and industrial rigging. Rope failure can result in equipment loss, environmental incidents, and fatalities. Current practice relies on periodic visual inspections by qualified personnel, a workflow that is labour-intensive, subjective, and difficult to scale. Yet the automation challenge is not simply one of classification. A deployable inspection system must simultaneously answer a range of interdependent questions: What type of damage is present? How severe is it? Is this presentation anomalous? What maintenance action is warranted? How is this rope deteriorating over time? Can a structured report be generated automatically? Answering all of these from a single image, without separate models for each task, is the core problem DART addresses. Existing automated rope inspection approaches treat these as isolated subproblems, a classification model here, a severity estimator there, with no shared representation that supports the full inspection workflow. Single-modality vision models fail on the hardest distinctions: a state-of-the-art ViT-H/14 backbone achieves only 54.76% accuracy on a 14-class rope damage benchmark because adjacent severity levels of the same damage type are visually indistinguishable without semantic context. Natural language provides exactly this context. A description such as “extensive surface abrasion with fibre bundle exposure along the outer sheath” encodes severity information that images alone cannot reliably convey across lighting conditions and viewpoints. 2 Related Work Automated inspection of SFRs has attracted growing interest driven by offshore mooring and floating wind turbine deployments 18 . Early systems relied on hand-crafted image features; a comprehensive survey by 36 identifies a persistent gap between single-task detectors and the multi-criteria decisions real inspections require. The same group introduced an instance-segmentation pipeline using Detectron2 37 and released the ROPE imagery dataset 35 that underpins DART’s benchmark. Complementary rope-health work includes statistical machine-learning models for tensile characterisation 14 and a lightweight neural network for real-time wire-rope tension monitoring via fibre Bragg grating sensors 42 . Defect detection has matured through convolutional and transformer architectures. 50 targeted object detection network in UAV, 11 developed transfer-learning pipelines for manufacturing defects, 48 used U-Net to detect surface damages on steel wire ropes (SWRs), and 33 combined residual networks and multi-channel feature fusion for detection of SWRs. Each of these addresses a single damage type or asset in isolation; DART instead learns a unified embedding space that supports all inspection decisions simultaneously. Transformer 44 was adapted to vision by 8 , who showed that patch-based self-attention matches or surpasses convolutional networks at scale. Subsequent self-supervised objectives have further narrowed the gap to supervised pre-training: 15 showed that masking a large fraction of patches and reconstructing pixel values yields strong representations, while 32 demonstrated that distillation across curated datasets produces dense visual features that transfer without fine-tuning. 3 established that a self-distilling student-teacher ViT learns semantically consistent patch features through local-to-global correspondences. DART builds on the Joint-Embedding Predictive Architecture (iJEPA) 1 , which replaces pixel-level reconstruction with prediction in latent space, producing more abstract patch representations suited to downstream transfer. 29 ; 19 provided theoretical bridges between JEPA and contrastive objectives, confirming that both paradigms optimise aligned information-theoretic quantities. DART’s CDD loss unites them: the JEPA reconstruction term targets structural features while the severity-aware InfoNCE 31 shapes the damage-severity manifold, with contrastive representation collapse further prevented by the MoCo-style EMA target encoder 16 . CLIP 34 established the blueprint for vision-language pre-training by contrastively aligning image and text encoders on web-scale pairs, yielding zero-shot transfer features competitive with supervised baselines. BLIP-2 23 and LLaVA 26 extended this to generative tasks by bridging frozen vision encoders with instruction-tuned language models. DART adopts Llama-3.2-3B-Instruct 28 as its text encoder, providing domain-adaptable instruction following at laboratory-scale training cost. In the inspection domain, 4 deployed a vision-language model for bridge crack classification conditioned on textual maintenance history, and 43 leveraged CLIP fine-tuning and prefix captioning to automatically generate safety observations. 19 proposed VL-JEPA, the closest architectural predecessor of DART, extending JEPA to joint vision-language prediction. Our SC-CMF module differentiates from VL-JEPA by introducing 14 per-class learnable gates that modulate language influence according to damage severity. 21 combined multi-gate mixture-of-experts 39 with JEPA for selective modality routing, an approach that motivated DART’s per-class gates, though DART implements them within a single cross-attention layer rather than a full routing table. 12 demonstrated JEPA-based models in safety-critical manufacturing via Gen-JEMA, and 47 applied cross-modal JEPA to remote sensing retrieval, DART applies the same philosophy to rope inspection with a four-term disentanglement loss absent from all prior JEPA variants. The MVTec AD benchmark 2 standardised unsupervised defect localisation evaluation and catalysed a progression from one-class SVMs to deep density-estimation methods. PaDiM 7 modelled per-patch feature distributions with multivariate Gaussians using pre-trained backbone features, and PatchCore 38 improved efficiency by subsampling a coreset of nominal embeddings and scoring anomalies by nearest-neighbour distance. A recent survey 27 confirms that foundation-model features increasingly replace purpose-trained autoencoders. DART’s Task 6 uses Mahalanobis distance 20 over its frozen backbone embeddings as the anomaly score, consistent with findings that pre-trained ViT features without fine-tuning are competitive on in-distribution shifts 17 . 45 further showed that attention-guided localisation improves interpretability on complex textures, directly relevant to spatially localised rope surface damage. Label scarcity is chronic in industrial inspection, where novel damage types may appear with only a handful of examples 46 . Prototypical Networks 40 classify by nearest embedding-space centroid and are used directly in DART’s Task 2, which evaluates frozen backbone prototypes at k\in\{1,5,10,20\} shots.