On the Status of Foundation Models for SAR Imagery

Nathan Inkawhich · 2025

Read On the Status of Foundation Models for SAR Imagery by Nathan Inkawhich (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21722.txt.

Introduction

Foundation models transformed natural image recognition, but radar imagery is a different world. Nathan Inkawhich investigates their viability for synthetic aperture radar object recognition in a 2025 paper. Self supervised frontier models adapt downstream with little labelled data and transfer well. Those strengths motivate applying them to SAR, where labels are scarce. Experiments first test today most powerful visual foundation models directly.

Models such as DINOv2, DINOv3, and PE-Core show shortcomings on SAR data in his tests. Radar images differ from natural images in speckle, geometry, and dynamic range. Robustness to distribution shift, a selling point elsewhere, cannot be assumed here. The paper takes stock of where foundation models stand for SAR. Its status assessment guides whether to adapt, retrain, or build domain specific models.

Catalog record

CREATORNathan Inkawhich
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-21722

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On the Status of Foundation Models for SAR Imagery Nathan Inkawhich Abstract In this work we investigate the viability of foundational AI/ML models for Synthetic Aperture Radar (SAR) object recognition tasks. We are inspired by the tremendous progress being made in the wider community, particularly in the natural image domain where frontier labs are training huge models on web-scale datasets with unprecedented computing budgets. It has become clear that these models, often trained with Self-Supervised Learning (SSL), will transform how we develop AI/ML solutions for object recognition tasks – they can be adapted downstream with very limited labeled data, they are more robust to many forms of distribution shift, and their features are highly transferable out-of-the-box. For these reasons and more, we are motivated to apply this technology to the SAR domain. In our experiments we first run tests with today’s most powerful visual foundational models, including DINOv2 [ 1 ] , DINOv3 [ 2 ] and PE-Core [ 3 ] and observe their shortcomings at extracting semantically-interesting discriminative SAR target features when used off-the-shelf. We then show that Self-Supervised finetuning of publicly available SSL models with SAR data is a viable path forward by training several AFRL-DINOv2s and setting a new state-of-the-art for SAR foundation models, significantly outperforming today’s best SAR-domain model SARATR-X. Our experiments further analyze the performance trade-off of using different backbones with different downstream task-adaptation recipes, and we monitor each model’s ability to overcome challenges within the downstream environments (e.g., extended operating conditions and low amounts of labeled data). We hope this work will inform and inspire future SAR foundation model builders, because despite our positive results, we still have a long way to go. 1 Introduction There has recently been significant effort in the AI/ML research community to develop foundational models. A dominant trend is the scaling of both model size and dataset size, along with the use of Self-Supervised Learning (SSL) objectives. In the image domain, we are now seeing training datasets on the order of several billion images and network architectures with billions of learnable parameters [ 2 , 3 ] (contrast this to yesterday’s ResNet-50s [ 4 ] with 25M parameters trained on 1M ImageNet images). These pretrained foundation models are then used for many diverse downstream tasks via several task-adaptation workflows (e.g., as fixed feature extractors or with some finetuning). For those in the Synthetic Aperture Radar (SAR) community, however, it remains an open question how applicable these public foundation models and associated techniques are to the SAR domain given that SAR data is not well-represented (if at all) in the web-based pretraining distributions. Thus, our goals in this work are to (1) update the community on the efficacy of existing foundational AI/ML feature extractors for exploiting SAR imagery; (2) suggest useful SAR-domain evaluation techniques and task-adaptation practices for use with this emerging model type; and (3) present a promising direction for the design of future SAR-domain foundation models which involves self-supervised finetuning. Since foundation models are a bit different than supervised ResNets of the past, we will now introduce some terminology that will be used throughout this paper. First, it helps to think of visual foundation models trained with SSL as task-agnostic feature extractors. They provide an embedding function for arbitrary images into a semantically-rich feature space where there are no explicit definitions of “classes.” This is notably different than task-specific models trained with supervised learning which seek to bin all input images into one of \mathcal{C} clusters, where \mathcal{C} is the number of classes in the fully-labeled training dataset. When we want to use a foundation model to solve a downstream task with \mathcal{C} classes, we instantiate and train a task-head to partition the model’s feature space to accomplish the specific task. A key advantage of well-trained foundation models is that often the feature extractor can remain fixed, and only the parameters of the task-head are learned. This significantly speeds up the task-adaptation process and often requires less labeled training data to learn the partition. However, as you will see in this paper, it is sometimes beneficial to update the parameters of the feature extractor along with the task-head. Since there are many ways one can perform this task-adaptation, we will use the term task-adaptation recipe to capture the steps and choices made during the process. Figure 1 : Feature space analysis between an off-the-shelf DINOv2 from torch.hub (left) and a custom SAR-finetuned DINOv2 (right) when embedding data from the SAMPLE (K=0) [ 5 ] task (done with t-SNE). As a preview of what’s to come, see Figure 1 . In this experiment, we investigate the feature space of two DINOv2 1 1 1 DINOv2 is widely considered one of the most powerful SSL visual foundation model techniques available [ 6 , 3 ] [ 1 ] foundation models when embedding data from the SAMPLE (K=0) task [ 5 ] , as visualized with t-SNE. Recall, SAMPLE involves the same 10 ground vehicle classes as MSTAR [ 7 ] and K=0 indicates the training data is fully synthetic while the test data is fully measured. On the plots, the orange markers are embeddings of the synthetic training data, the blue markers are embeddings of the measured test data, and the different marker shapes indicate different classes. Finally, the embedding on the left subplot uses an off-the-shelf pretrained DINOv2 from torch.hub while the right subplot uses a custom SAR-finetuned DINOv2 we train in this work. The difference is obvious. The torch.hub DINOv2 (which has never been trained on a SAR image) produces an unusable feature space for this task which separates the data by synthetic vs. measured instead of by class. The SAR-finetuned DINOv2 produces a more nuanced and discriminative feature space which is separable by class and also co-locates many of the synthetic and measured representations of each class even though it has never been trained on a labeled SAR image. As will be discussed in more detail in the following sections, the implications of this observation are two-fold. First, even the best publicly available foundation models from the natural image domain are not effective in the SAR domain off-the-shelf and require expensive task-adaptation recipes to be competitive downstream. Second, SSL in the SAR domain is a promising approach to creating powerful SAR foundation models and these models can be used downstream with cheaper adaptation recipes. Overall, our contributions in this work are as follows: • We provide a much needed update on the functionality of foundational AI/ML models for exploiting SAR data which includes today’s most powerful visual models; • We highlight the importance of specifying the downstream task-adaptation recipe and show that there are several ways one can adapt a model depending on the suitability of the base model as well as compute and data limitations in the downstream environment; • We make new comparisons between today’s s.o.t.a. models from natural imagery (e.g., DINOv2 [ 1 ] and DINOv3 [ 2 ] ) and several SAR-domain specific methods (e.g., SARATR-X [ 8 ] ); • Finally, we show a promising path forward for developing future SAR foundation models based on self-supervised finetuning DINOv2 with SAR data. 2 Background & Related Work Visual Foundation Models. By far, most

Read the complete text