A Zero-Shot Learning Approach for Ephemeral Gully Detection from Remote Sensing using Vision Language Models
Seyed Mohamad Ali Tousi, Ramy Farag, Jacket Demby’s, Gbenga Omotara, John A. Lory, G. N. DeSouza · 2025
Read A Zero-Shot Learning Approach for Ephemeral Gully Detection from Remote Sensing using Vision Language Models by Seyed Mohamad Ali Tousi, Ramy Farag, Jacket Demby’s, Gbenga Omotara, John A. Lory, G. N. DeSouza (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01169.txt.
Introduction
Seyed Mohamad Ali Tousi and co-authors present computer pipelines that spot short-lived erosion gullies in farm images without any training examples, in a research paper from 2025. These gullies are a leading cause of soil loss, and early spotting helps farms act sooner. No earlier work had automated the task well.
The pipelines use vision-language models that read images and text together. On test images they passed seventy percent accuracy, with strong scores for true gully finds. The team also released the first public dataset for the task, labeled by soil and plant experts. Their pipelines beat a transfer-learning rival, showing zero-shot methods can serve farm monitoring.
Catalog record
| CREATOR | Seyed Mohamad Ali Tousi, Ramy Farag, Jacket Demby’s, Gbenga Omotara, John A. Lory, G. N. DeSouza |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01169
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Zero-Shot Learning Approach for Ephemeral Gully Detection from Remote Sensing using Vision Language Models Seyed Mohamad Ali Tousi, Ramy Farag, Jacket Demby’s, Gbenga Omotara, John A. Lory, G. N. DeSouza Abstract Ephemeral gullies are a primary cause of soil erosion and their reliable, accurate, and early detection will facilitate significant improvements in the sustainability of global agricultural systems. In our view, prior research has not successfully addressed automated detection of ephemeral gullies from remotely sensed images, so for the first time, we present and evaluate three successful pipelines for ephemeral gully detection. Our pipelines utilize remotely sensed images, acquired from specific agricultural areas over a period of time. The pipelines were tested with various choices of Visual Language Models (VLMs), and they classified the images based on the presence of ephemeral gullies with accuracy higher than 70% and a F1-score close to 80% for positive gully detection. Additionally, we developed the first public dataset for ephemeral gully detection, labeled by a team of soil- and plant-science experts. To evaluate the proposed pipelines, we employed a variety of zero-shot classification methods based on State-of-the-Art (SOTA) open-source Vision-Language Models (VLMs). In addition to that, we compare the same pipelines with a transfer learning approach. Extensive experiments were conducted to validate the detection pipelines and to analyze the impact of hyperparameter changes in their performance. The experimental results demonstrate that the proposed zero-shot classification pipelines are highly effective in detecting ephemeral gullies in a scenario where classification datasets are scarce. 1 Introduction Soil erosion is a geomorphological land degradation process that can result in environmental harm, property damage, loss of livelihoods and services, and social and economic challenges. Ephemeral gully erosion reduces the sustainability of agricultural systems and causes significant sediment-related issues downstream [ 1 ] . Ephemeral gullies remain a prominent and hard-to-treat cause of soil erosion [ 2 ] . A fast, reliable and cost-effective method to detect ephemeral gullies in the agricultural fields could play a crucial role in preventing the soil erosion and successful execution of soil conservation practices. A detection system using remote sensing images to distinguish between agricultural fields that show signs of ephemeral gully formation versus ephemeral gully-free agricultural fields will advance efforts of soil researchers, farmers and responsible agencies to study, treat and/or prevent soil erosion. The emergence of Foundation Models, Vision Language Models (VLM), and Visual-Question-Answering (VQA) has great potential for tackling a wide range of computer vision challenges, particularly in remote sensing ( [ 3 , 4 , 5 ] ). When classifying nuanced and subjective concepts, Foundation Models and VLMs tend to hallucinate. To address this problem, techniques such as Modeling Collaborator (MC) [ 6 ] have been introduced. One of the advantages of MC is that it does not require an extensive amount of labeled data which is especially attractive to us since labeled datasets for ephemeral gully detection are extremely scarce. The lack of sufficient datasets for detecting ephemeral gullies to train conventional computer vision models leads to the exploration of methods that do not rely heavily on extensive labeled data. Techniques such as zero-shot classification, and transfer learning are common alternatives. We assessed SOTA open-source zero-shot, transfer learning, and VLM-based classification methods in the task of classifying remote sensing images with signs of ephemeral gullies in agricultural fields. Additionally, we are providing the first dataset for detecting ephemeral gullies using remote sensing images, labeled by a group of soil and plant science experts. Our key contributions are as follows: 1. We present and evaluate three classification pipelines for detecting ephemeral gullies in remote sensing images, exploring zero-shot, transfer learning, and Vision-Language Models (VLMs). To our knowledge, this is the first work to successfully address this under-explored task. Inspired by [ 6 ] , we further integrate large language models (LLMs) to enhance reasoning in the classification process, using VLMs for Visual Question Answering (VQA) and LLMs for reasoning, and compare their performance against existing zero-shot classification baselines. 2. We conduct extensive experiments to investigate the choices of VLMs, types and number of questions, and aggregation methods for VQAs. Those experiments lead to a better understanding of the advantages and limitations of zero-shot classification methods such as [ 6 ] . 3. We make available the first public dataset for the detection of ephemeral gullies using remote sensing images. The dataset comprises high-resolution remote sensing RGB images of agricultural locations over different periods of time. The images are labeled by a team of soil and plant science experts to establish the ground truth classifications. 2 Background and Related Work In this section, we will explore the existing research articles in the literature on ephemeral gully detection (assuming its presence in the image), and ephemeral gully formation assessments using remote sensing imagery. We also explore the SOTA zero-shot classification methods and VLMs that form the foundation of this research. Figure 1 : The proposed independent pipelines to detect ephemeral gullies. We are exploring three types of Pipelines; (A) : Feeding the remote sensing RGB images directly to SOTA zero-shot classification methods such as CLIP [ 7 ] , CuPL [ 8 ] , and a variety of VLMs including Llama 3.2-Vision [ 9 ] , Qwen [ 10 ] , and Llava [ 11 ] . (B) : Inspired by [ 6 ] , we pass the RGB images through a VQA system, which responds to a series of Yes/No questions regarding visual attributes indicative of ephemeral gullies. An LLM then aggregates these responses to classify the images. (C) : Similar to (B), we propose asking the VLM a single descriptive question about the image, with an LLM interpreting the response to make a final classification. 2.1 Ephemeral Gullies in Remote Sensing Remote detection of ephemeral gully through classification remains largely unexplored, though manual assessment and evaluation are well-documented in agricultural literature [ 12 , 13 , 14 , 15 ] . However, the time commitment associated with manual assessments limits their scalability for large-scale applications. Authors in [ 16 ] tested the effectiveness of using Google Earth temporal images to assess ephemeral gullies in U.S. agricultural fields, presenting a systematic method for identifying and locating these gullies within a certain error margin. Their approach relied heavily on manual labor and utilizes Google Earth images, which lack the image standardization needed for remote sensing in computer vision contexts. Some researchers have attempted to automate parts of the process. There is extensive research on ephemeral gully risk assessment using topographic index models to identify areas of the field most susceptible to formation of ephemeral gullies [ 17 ] . There are examples of computer vision techniques applied to detect ephemeral gullies. For example, [ 18 ] employed conventional computer vision techniques, such as directional edge detection, to create a ”semiautomatic” method for identifying ephemeral gullies. Manual pre-identification of areas containing gullies was necessary due to the faint visibility of these features. Similarly, [ 19 ] applied deep learning methods like Convolutional Neural Networks (CNNs) [ 20 ] and U-Nets [ 21 ] for delineating gullies in non-agricultural areas. Current model- and image-based methods require either post-identification verification of presence of a gully or manual pre-identification of gully-prone regions. To date, the