FrogDogNet: Fourier frequency Retained visual prompt Output Guidance for Domain Generalization of CLIP in Remote Sensing

Hariseetharam Gunduboina, Muhammad Haris Khan, Biplab Banerjee · 2025

Read FrogDogNet: Fourier frequency Retained visual prompt Output Guidance for Domain Generalization of CLIP in Remote Sensing by Hariseetharam Gunduboina, Muhammad Haris Khan, Biplab Banerjee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16433.txt.

Introduction

Hariseetharam Gunduboina and co-authors improve scene sorting from satellite photos in this 2025 paper. Large image-text models can sort scenes with no task training, but they falter when a new area looks unlike their training data. Past fixes use whole-image traits, which add noise from cluttered backgrounds.

Their model keeps only the steady, low-detail parts of the image signal and drops noisy detail. It first picks out strong traits with a focus step, then filters them before building text prompts. Tests on four data sets and three shift tasks beat prior prompt methods throughout. The authors argue that keeping steady structure is the key to sorting scenes in new places.

Catalog record

CREATORHariseetharam Gunduboina, Muhammad Haris Khan, Biplab Banerjee
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-16433

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FrogDogNet: Fourier frequency Retained visual prompt Output Guidance for Domain Generalization of CLIP in Remote Sensing Hariseetharam Gunduboina, Muhammad Haris Khan, Biplab Banerjee Abstract In recent years, large-scale vision-language models (VLMs) like CLIP have gained attention for their zero-shot inference using instructional text prompts. While these models excel in general computer vision, their potential for domain generalization in remote sensing (RS) remains underexplored. Existing approaches enhance prompt learning by generating visual prompt tokens but rely on full-image features, introducing noise and background artifacts that vary within a class, causing misclassification. To address this, we propose FrogDogNet , a novel prompt learning framework integrating Fourier frequency filtering and self-attention to improve RS scene classification and domain generalization. FrogDogNet selectively retains invariant low-frequency components while eliminating noise and irrelevant backgrounds, ensuring robust feature representation across domains. The model first extracts significant features via projection and self-attention, then applies frequency-based filtering to preserve essential structural information for prompt learning. Extensive experiments on four RS datasets and three domain generalization tasks show that FrogDogNet consistently outperforms state-of-the-art prompt learning methods, demonstrating superior adaptability across domain shifts. Our findings highlight the effectiveness of frequency-based invariant feature retention in generalization, paving the way for broader applications. Our code is available at https://github.com/HariseetharamG/FrogDogNet 1 Introduction Figure 1 : Impact of Fourier Filtering on RS Image Analysis: (a) demonstrates that retaining 50\% of low-frequency components (LFCs) preserves structural details while reducing noise in the frequency magnitude spectrum. (b) presents a sensitivity analysis, showing that keeping 350 out of 512 LFCs of visual features f_{v}(x) achieves the highest average generalization performance. Remote Sensing (RS) image analysis underpins a diverse range of tasks including Earth observation DATA 2024 , environmental monitoring Sabins 1999 ; Hino et al. 2018 , urban planning Li et al. 2024b , and disaster management Inglada and Giros 2004 ; Van Westen 2000 , to name a few. Although deep learning models have substantially advanced RS analytics Ma et al. 2019 , they often exhibit performance drops under domain shifts—variations in sensor modalities, atmospheric conditions, and geographical characteristics that cause discrepancies between training (source) and testing (target) data distributions. Two core strategies address such shifts: domain adaptation (DA), which aligns source and target distributions during training Tuia et al. 2016 ; Ganin and Lempitsky 2015 ; Farahani et al. 2021 ; Saha et al. 2022 , and domain generalization (DG), which strives for robust performance in unseen target domains Duraklı and Aptoula 2023 ; Li et al. 2018a ; Zheng et al. 2021a ; Zhou et al. 2022a . DG is crucial in RS due to the difficulty of collecting labeled target-domain data. Challenges include cross-sensor variability, seasonal and atmospheric changes, geographic distribution shifts, and limited real-time data for disaster monitoring Tuia et al. 2016 ; Zhu et al. 2017 ; Morley et al. 2018 ; Hu and Tang 2021 . Despite progress via adversarial training Martini et al. 2021 , meta-learning Yang et al. 2024 , and domain-invariant representations Yaoting 2023 ; Ghanbarzadeh and Soleimani 2024 , existing DG approaches struggle to handle the extreme spectral and spatial variations in RS datasets. Beyond domain shifts, RS tasks typically suffer from limited annotated data, as labeling requires domain expertise (e.g., differentiating land-use categories or vegetation types). The substantial cost and logistical complexity of labeling across diverse environments hamper large-scale data curation, exacerbating the challenge of building robust models. Few-shot learning (FSL) Russwurm et al. 2020 ; Zhang et al. 2021 ; Ji et al. 2022 ; Finn et al. 2017b ; Chen et al. 2019 partially addresses this issue by enabling models to learn from minimal supervision. Although FSL has been explored for multispectral and hyperspectral RS imagery Xue et al. 2022 ; Alajaji et al. 2020 ; Liu et al. 2018 ; Pal et al. 2021 , most methods remain image-centric, inadvertently capturing extensive background clutter in RS scenes, such as oceans, deserts, or forests. Since key objects of interest (e.g., urban structures, agricultural plots) are often small and spatially scattered, Standard FSL pipelines optimized for natural images often generalize poorly when applied to RS datasets. This is validated in our work, where methods like CLIP Radford et al. 2021a , CoOp Zhou et al. 2022c and CoCoOp Zhou et al. 2022b struggle with RS data. Vision-Language Models (VLMs) like CLIP Radford et al. 2021a , ALIGN Jia et al. 2021b , Florence Yuan et al. 2021a , and LiT Zhai et al. 2022 leverage textual supervision for powerful zero-shot adaptation. Adaptations for RS (e.g., RS-CLIP Li et al. 2023b , RemoteCLIP Liu et al. 2024 , and ChangeCLIP Dong et al. 2024 ) illustrate the promise of combining visual and textual cues but remain challenged by large homogeneous backgrounds and small, high-value objects in RS. Although prompt-learning techniques Zhou et al. 2022c ; Zhou et al. 2022b ; Zhu et al. 2022 ; Gao et al. 2021 ; Khattak et al. 2023 ; Bose et al. 2024 have been proposed to refine VLMs for downstream tasks, most still operate on entire images, rendering them susceptible to overfitting on dominant yet irrelevant features. Multi-scale methods Singha et al. 2023 ; Bhattacharya et al. 2023 address some of these challenges but often lack a robust mechanism to filter out noise and irrelevant background artifacts. This limitation reduces their effectiveness, particularly under varying spectral conditions where such artifacts can significantly degrade performance. To address these gaps, we propose leveraging Fourier Transform-based filtering to enhance DG for RS image analysis. Fourier filtering is widely employed in RS for denoising and spectral feature extraction, as it separates low-frequency semantic structures from high-frequency noise and sensor artifacts. This property reduces sensitivity to environmental variations and promotes stable feature extraction. However, filtering out high-frequency components too aggressively can discard fine-grained details critical for object boundaries. To mitigate this, we introduce a self-attention mechanism Vaswani et al. 2017 that selectively retains local cues while preserving robustness under domain shifts. As illustrated in Figure 1 (a), retaining the top 50\% low-frequency components (LFC) effectively removes high-frequency clutter but can risk losing valuable details. Our analysis in Figure 1 (b) indicates that retaining 350 out of 512 LFCs yields the best average performance across multiple RS datasets and generalization tasks. We introduce a FrogDogNet (Fourier frequency Retained visual prompt Output Guidance for Domain Generalization of CLIP in Remote Sensing) framework for few-shot RS recognition, integrating a Fourier Filter Block (FFB) within a CLIP-based pipeline. The FFB focuses on semantically relevant structures while discarding background clutter, thereby improving domain generalization. A Prompt Alignment Loss aligns learned prompts with RS-specific initializations, bolstering transferability across diverse imaging conditions. To further refine feature representations, we design a lightweight Meta-Net that converts filtered embeddings into tokens optimized for downstream tasks. Our principal contributions are threefold: - Fourier Filter Block (FFB) : We propose a dedicated block that isolates essential low-frequency RS features, suppresses noise, and, combined with self-attention, selectively preserves crucial object boundaries. - Remote Sensing Prompt Alignment Loss : We introduce a novel loss function that harmonizes learned prompts with various RS-specific text initializations, significantly enhancing zero-shot and few-shot adaptability. - Comprehensive Evaluation : Extensive experiments on multiple RS benchmarks—including base-to-new class generalization, cross-dataset transfer, and multi-target adaptation—demonstrate

Read the complete text