JVLGS: Joint Vision-Language Gas Leak Segmentation
Xinlong Zhao, Qixiang Pang, Shan Du · 2025
Read JVLGS: Joint Vision-Language Gas Leak Segmentation by Xinlong Zhao, Qixiang Pang, Shan Du (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-19485.txt.
Introduction
Xinlong Zhao, Qixiang Pang, and Shan Du joined vision and language in 2025 to find gas leaks in infrared video. Leaks are dangerous and must be caught early, but their plumes are blurry, translucent, and constantly changing form, which makes pixel-level splitting unreliable. Their method, JVLGS, feeds textual accounts beside the infrared imagery so the two modalities correct each other's weaknesses when outlining a plume.
Because leaks are sporadic, most frames contain nothing at all, and rival systems waste their errors on false alarms from noise and innocent objects. A self-tuning post-processing module in JVLGS is built specifically to suppress those phantom detections. Across varied factory cases the method greatly outperformed the leading splitting methods. And unusually, it stayed strong both when trained with full supervision and when given only a few examples, settings where competitors typically manage only one.
Catalog record
| CREATOR | Xinlong Zhao, Qixiang Pang, Shan Du |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-19485
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
JVLGS: Joint Vision-Language Gas Leak Segmentation Xinlong Zhao, Qixiang Pang, Shan Du Abstract Gas leaks pose serious threats to human health and contribute significantly to atmospheric pollution, drawing increasing public concern. However, the lack of effective detection methods hampers timely and accurate identification of gas leaks. While some vision-based techniques leverage infrared videos for leak detection, the blurry and non-rigid nature of gas clouds often limits their effectiveness. To address these challenges, we propose a novel framework called Joint Vision-Language Gas leak Segmentation (JVLGS), which integrates the complementary strengths of visual and textual modalities to enhance gas leak representation and segmentation. Recognizing that gas leaks are sporadic and many video frames may contain no leak at all, our method incorporates a post-processing step to reduce false positives caused by noise and non-target objects—an issue that affects many existing approaches. Extensive experiments conducted across diverse scenarios show that JVLGS significantly outperforms state-of-the-art gas leak segmentation methods. We evaluate our model under both supervised and few-shot learning settings, and it consistently achieves strong performance in both, whereas competing methods tend to perform well in only one setting or poorly in both. We publish our code at: https://github.com/GeekEagle/JVLGS . 1 Introduction Industrial gas leaks constitute critical emergencies that can lead to severe accidents. Methane contributes significantly to the greenhouse effect, while others like acetaldehyde are highly toxic [ 3 ] . Detecting these leaks is especially challenging because most industrial gases are colorless and odorless, making them imperceptible to the human senses. Traditional detection methods rely on electrochemical or photoionization sensors to monitor pipeline conditions [ 53 , 1 ] . These approaches are hazardous because inspectors should place themselves near suspected leak points. To address these limitations, vision-based detection methods have gained increasing attention for their non-contact nature and potential for automation [ 51 , 38 ] . These methods not only reduce labor costs but also eliminate direct exposure risks for inspectors [ 33 ] . Among these methods, video surveillance integrated with infrared (IR) imaging has shown strong potential for detecting gas leaks. Nevertheless, video surveillance systems still face significant challenges. Even with infrared cameras, gas leaks are difficult to find, as they blend seamlessly into the backgrounds of frames. To address this issue, Lu et al [ 24 ] proposed a Gaussian-based method for background modeling to enable adaptive gas leak segmentation. However, its performance degrades significantly in the presence of irregular gas shapes and dynamic backgrounds. Similarly, optical flow-based methods [ 49 , 41 ] are limited by the inconsistent shapes of gas and susceptible to noise interference. Recent advances in deep learning have led to significant progress in camouflaged object detection (COD) tasks [ 43 ] . For instance, SINet-V2 [ 10 ] first outlined the rough contours of concealed objects. Wang et al [ 39 ] , leveraging the YOLO series model [ 36 ] , demonstrated strong performance in detecting visible, non-rigid objects like dust. GasSeg [ 52 ] integrated convolutional neural networks (CNNs) and Vision Transformers (ViT) [ 9 ] to develop a lightweight model for gas leak segmentation. However, its performance is limited by the inability to effectively capture motion cues from faint and ambiguous targets. While motion-based techniques offer a promising direction, long-term motion techniques [ 4 , 58 , 6 ] encounter difficulties due to the highly variable shapes of gas leaks and the lack of consistent features for the model to track over extended periods. In contrast, short-term approaches, such as RCRNet [ 48 ] , leverage optical flow to capture motion characteristics. Nevertheless, both long-term and short-term methods remain susceptible to noise interference. Other recent work, such as SLT-Net [ 7 ] adopted implicit motion analysis and showed potential feasibility. However, it struggles to distinguish gas leaks from texturally similar objects, such as clouds. FGSTP [ 57 ] introduced fine-grained perception to refine motion information. However, the extracted motion features may not be reliably correlated with the actual target objects. One potential approach to enhance the feature representation is to integrate temporal-spatial models with Vision-Language Models (VLMs) [ 54 ] . CLIP [ 31 ] is one of the most well-known VLMs, employing a ViT as both its image and text encoder. However, its inference is too slow for real-time object segmentation. VisualBERT [ 21 ] , designed for end-to-end vision-language correlation learning, offers flexibility but lacks specialization for spatially precise tasks. DALL-E [ 32 ] combines visual information with Large Language Models (LLMs) [ 56 ] to generate creative outputs, but it is not suited for accurate segmentation. OWLv2 [ 26 ] uses a ViT to process both image and text inputs by self-training, showing potential for open-vocabulary object detection. However, to the best of our knowledge, few existing methods effectively incorporate text-guided segmentation for blurry, non-rigid objects like gas leaks. In this paper, we propose Joint Vision-Language Gas leak Segmentation (JVLGS), a novel framework designed to enhance model generalizability across diverse gas leak scenarios. Our method integrates text prompts between the vision encoder and motion extractor to enrich the information of the encoder. Following vision-language integration, the designed temporal-spatial module uses these enhanced features to analyze the motion and spatial cues of gas leaks. The decoder then integrates the multi-scale information through upsampling to generate the final masks. Many prior methods [ 29 , 7 , 57 ] are vulnerable to noise and minor movements in non-leak scenes, often producing false positives when no target is present. To address this, we incorporate a post-processing step to eliminate these false positives, ensuring entirely black masks for non-leak cases. We evaluate our model on two benchmark datasets under both supervised and few-shot learning settings to reflect realistic video surveillance use cases. Our main contributions are summarized as follows: • We propose JVLGS, a model that integrates manually crafted text prompts with a video clip to enhance feature extraction. These prompts enable accurate localization and detection of gas leaks. • We design a lightweight vision-language fusion approach, enabling more effective integration of visual features with textual representations. • Our proposed model achieves significant performance gains in both supervised and few-shot learning, effectively simulating real-world leak detection scenarios. 2 Related Work 2.1 Gas Leak Detection and Segmentation Traditional gas leak detection and segmentation methods rely on manual sampling and laboratory analysis using chemical reactions [ 2 ] . They are limited by slow response times, high labor costs, and risks associated with manual gas sampling [ 60 ] . Advancements in infrared thermography have emerged as a promising alternative, offering improved accuracy, broader detection coverage, and visualized gas leaks. However, automatically segmenting invisible gas is still full of challenges. Existing methods can be categorized into image-based and video-based approaches. Image-based methods [ 45 , 11 , 50 ] rely on local features, such as shape and texture, but they often fail in blurry scenarios. In contrast, video-based methods [ 37 , 19 , 5 ] leverage the motion characteristics of gas in video sequences, resulting in improved accuracy. However, these methods often involve high computational costs and are not well-suited for real-time segmentation. To overcome these limitations, we propose a video-based segmentation network that achieves SOTA performance while maintaining the efficiency